Let’s build an AI app – pt. 2

As AI continues to make headlines, with Mistral releasing their first multi-modal model and OpenAI releasing a “reasoning” model, becoming an AI developer continues to be at the forefront of a lot of people’s minds. And for most developers, knowing how to go from nothing to a functional website is the goal, so that’s what I’m here to attempt to teach you. If you’re already developing AI apps, I would love to hear about the tools you’re using, what you’re trying to do, and especially anything you’re struggling with.

If you want the quick and dirty of it – here you’ll find a demo and RedisVL data loader. In the last week I went from searching images of strawberries and returning the name of the closest images to searching images from anime posters and returning both the actual closest matched image and the name of the anime it came from. Below you’ll see that a search for “swords” brings up the poster for Mushoku Tensei: Jobless Reincarnation, which prominently features swords clashing.

How’d we get here?

So last week I started looking for a dataset. If you’re getting into data science or machine learning – you’ve probably already heard of Kaggle and that’s where I started. I was thinking about some recommendations I’ve had for anime recently, and so started searching for anime datasets.

A lot of datasets that you come across may have some things you want and some things you don’t care about. If you find a dataset that meets all your needs and is interesting to you, that’s great! There are plenty of them out there that are fundamentally interesting! However, there are also a lot of ways that you can create your own dataset if you can’t find one that suits your needs.

This is all to say, if you can’t find a dataset that excites you, make one yourself!

So you know have some data, now what?

Well then you have to store the data. I considered vectorizing the data and loading it into Redis as I went, this was mainly driven by the images I gathered. Because csv files are one of the easiest ways to pass textual information between Python scripts (in my humble opinion, feel free to defend pickle files in the comments), I was planning on writing all my data to a csv. But csv files only store text. So I either would have to go ahead and vectorize the image to avoid saving it locally intermediately, or save the image and put the image name in the csv for processing later.

While the “go ahead and vectorize it” seemed appealing to me, there were quite a few times where the script just hung for a minute or two before I manually killed the program. Only to change absolutely nothing and have it run perfectly 20 seconds later. Other times something that I thought I had successfully processed would suddenly break, meaning that I also had to kill my Redis connection and clear out my database after any failed run. Since I’m still building, I definitely don’t want to accidentally think something is working when it isn’t really. So, that means one script for scraping data and one for loading and querying it.

After getting the data how I want it, let’s make it searchable.
RedisVL uses a schema to define how your data looks, and for what I have currently, this is how my schema looks:

schema = {
   "index": {
       "name": "anime_demo",
       "prefix": "anime",
   },
   "fields": [
       {"name": "title", "type": "text"},
       {"name": "episodes", "type": "numeric"},
       {"name": "rating", "type": "numeric"},
       {"name": "short_description", "type": "text"},
       {"name": "tags", "type": "tag"},
       {"name": "poster_vector", "type": "vector",
           "attrs": {
                "dims": 768,
                "distance_metric": "cosine",
                "algorithm": "flat",
                "datatype": "float32"
           }
        }
   ]
}

We’ll have a title, the number of episodes, the numeric rating, a short description, tags, and a vector embedding of the poster image. The dimensions of your vector will change based on what model you’re using – but knowing that information and correctly passing it along to Redis are critical steps to make vector search work correctly.

Something to call out here is that not all data is stored the same way everywhere. While the strings contained special characters for Shōnen and Shōjo, writing and reading them using Python’s csv library didn’t seem to preserve the ō, meaning at first I had some hex representations to deal with. So while it is also something I’ll be exploring this next week, I decided to just handle making the tags ‘Shounen’ and ‘Shoujo’ explicitly. This can definitely impact data quality, and understanding why you may need to clean data is just as important as doing it correctly.

Aside from a bit of processing, including the easy one line command to get the vector representation for the images, the data loader is simply tying the fields from the csv to the fields of the schema I defined.

After that the changes to the Gradio portion were pretty simple. I changed up the structure a bit due to needing to pass the image in a format that would work for Gradio. So now we have a demo that contains the elements for the webpage and the search functionality, then we launch it.

with gr.Blocks() as demo:
   search_term = gr.Textbox(label="Search the top 100 anime by their 
      posters")
   search_results = gr.Textbox(label="Closest Anime (by poster 
      search)")
   poster = gr.Image(label="Closest Anime Poster")


   def anime_search(text):
       embedding = vectorizer.embed(text, as_buffer=True)
       query = VectorQuery(vector = embedding, vector_field_name = 
           "poster_vector", return_fields=["title", "img_name"])
       results = index.query(query)[0]
       title = results['title']
       img = results['img_name']
       return title, Image.open(f'anime_posters/{img}')

   gr.Button("Search").click(fn=anime_search, inputs=search_term, 
      outputs=[search_results, poster])


demo.launch()

Things I’ll do next

I’m still on the hunt for a bit more data about these anime – I would love a longer description and potentially more images. I definitely want to see how many I can add, and how many fields can be vectors, in a free tier Redis database.

I’m also planning to see what all Gradio can do – ideally I’d like to be able to cycle through the results and display both the poster and the anime title for about the top 5 results from the vector search.

The other thing I’m going to do this week is a bit of refactoring. I’d like to add some of the standard best practices to this repository, like a real README and a requirements.txt that will help you try it out if you want.

I would love to hear from you about the AI apps you’re building, if you know how to scroll through images in Gradio, and if you have favorite tools for scraping dynamic web content. Check out the RedisVL project yourself if you’re considering an AI app backed by Redis, as it really does make this whole thing so much easier.

Things I learned this week

1. There’s an upper limit to how complex a Redis search query can be before it ends up blowing the stack and becoming a syntax error. While I, nor you, will likely ever encounter this error, some of the wrapper libraries exacerbate this problem by being heavy on the parentheses. So if you’re ever doing a query with something like 100 ORs, once you start getting syntax errors you should pause and reflect on how you could be doing it more efficiently.
2. OpenAI’s safety committee will be overseeing the security and safety processes for their AI models development and deployment as an independent body, seemingly with the goal of an independent AI oversight board that will enable the sharing of cybersecurity and general threat information.
3. Intel is going to be making a custom chip for Amazon.