#6 - Unlocking Business Value with Vector Databases – A Deep Dive

This episode of the AI Demystified for Executives podcast, presented by Dragonfly Risings School of Data and hosted by Andrew Psaltis, dives deep into the concept of vector databases. Discussing their importance in applying AI to business contexts, the episode explores the fundamentals of vector databases, their embedding models, and their applications across various industries. It highlights the increasing relevance of vector databases in AI-powered search and personalization, the impact of unstructured data, and the semantic meaning preserved in vector embeddings. Practical business use cases such as Walmart's search capabilities and Morgan Stanley's AI assistant are illustrated, along with potential challenges and considerations for integrating vector databases into existing systems. The episode emphasizes the transformative potential of vector databases in enhancing customer and employee experiences through improved search and recommendation systems.

Transcript

 Dragonfly Risings school of data proudly presents the AI demystified for executives podcast. This is the podcast for executives who want to learn how to apply aI to their business. I'm your host. Andrew Psaltis 

be sure to subscribe to this podcast on your favorite platform, where you listen to others. Without further ado, let's dive into this episode. 

Welcome back. If you listen to Monday's quick bites. Where we talked about vector databases. Just from the high level of vector search.  This is our Wednesday deep dive.  We're going to go. Deeper. Into vector databases. And really try to understand their increasing relevance in AI. e-commerce and customer support context, This coincides, with the rise of chat, GPT, like tools. AI powered search. And consumers increasing demand for personalization. Forester. Just released in the beginning of October. There first. Forrester wave for vector databases. This is a pretty significant milestone for this category in the industry. And Forrester anticipates. That by the end of 2026. Most organizations will have vector databases in production. 

So a lot of the attraction. Vector databases. Is due to the rise of generative AI. Lots of new use cases. So it's really important as you think about it. Of how. To think about your organization. Using this type of technology. And this is going to be across all different industries. Financial services, retail, healthcare, manufacturing, energy. All different industries will see an uptick. In the use of vector databases. 

So let's go a little bit deeper and really what our vector databases. 

When you think of data? You probably think of data in say a spreadsheet. Really neatly organized numbers in the spreadsheet. This type of data is called structured data. Because it could easily be stored in a tabular format. Whether that's saying Google sheets and Excel. And whatever tool that you're using, you could think about storing data in rows and columns. 

But roughly 80% of today's data. Is actually unstructured. So unstructured data. That's the images. Taxed. So documents, social media posts, emails. Time series data. And that's not just sensors. But also video audio. 



 With this type of data. It's difficult to store in an organized way. So you could enable users to easily find what they're looking for. 

If you think about how would you store. Audio files. hard to put them into. Rows and columns. The same thing with , all the photos that you may have taken on your phone. How would you think about storing those in rows and columns? It'd be really hard to do that. To allow. User to search for similar images.  Similar to  how on your phone? You could search for images by a person. And buy a place. 

 Artificial intelligence and machine learning innovations. Over the last number of years have enabled us. To come up with really interesting ways to numerically represent. This unstructured data. Without losing its semantic, meaning. And doing this as what's called a vector embedding. And vector embeddings are core to a vector database. 

Vector embedding is just a long list of numbers. Each describing a feature of the data. 

You think about. Doing. Facial recognition. 

 If you wanted to. Identify faces that are similar to another one. You would need to put this into a vector in bedding. Like eye color, hair, color, distance between the eyes. No size ear size, skin tone. 

With that, then you could find other faces that are close to this one. think of a vector in bedding. As just a long list of numbers. That describe a feature of the data you're wanting to store. So if it's for facial recognition, There'd be an embedding for. I color. Hair color. Distance between the eyes, no size ear size, skin tone, et cetera. With that. You are now searching for vectors that are similar to another one. 

Similar to. Matthew May have done In high school. Or at some time during your academic career. I had RV you're searching of where are two vectors close to each other? Doing a proximity. 

Yeah. So vector databases. Really Excel. At. Handling. 

For any data? Where you want to have that context? And you keeping that context and storing at a list of. 

Numbers that represent. The context of. That data. If it's a sentence. We're representing the contextual meaning. Of a word. And you may store a word. As a vector. 

So it's really important concept here to just remember that these Vectren beddings and the real power of them besides converting. An image to numbers. A word, a sentence, a document to a list of numbers. The real key thing is that. They're capturing the semantic meaning. Of these objects. In relation to other objects. 

So when you think about that, if you have two vector embeddings, two long list of numbers, You're able to easily do. Distance calculations to see how close are they to another. If you imagined a. 

Scatter plot. And on that you had dog. You had cat. You had. Alligator. And you had airplane. 

You could imagine that in dog, you may have underneath there. The different breeds. So say it's a lab, it's a pit bull. It's a golden retriever. Those are going to be closer in that scatterplot. More similar to each other. Then airplanes were maybe Airbus or Boeing. They may appear in a different part of that scatterplot. So now when you're looking for. A dog in a query. You're going to end up being. Closer to. Labrador pet Paul, the different breeds. That are grouped. 

in that part of the plot. Okay. So you're just matching these things together and we're able to group these. 

And this could be done as well for product descriptions. For documents. So let's imagine that we had a product description. For example, lightweight laptop for travelers. This might get converted to a vector that is point 32. 1.78. Point five, 6.12. Each number in that vector. So the point 32 at 1.78. The 0.56, the 0.12. Each one corresponds to semantic features. Derive from the text. Such as the context, the topic or the sentiment. 

Okay, so just. A very important concept here. Of these vector embeddings. You'll hear this referred to as this high dimensional space. these vectors live in a multi-dimensional space. So when I say the, imagine on a scatterplot, it's really more appropriate. To think of them like a 3d graph. 

But there could be hundreds or thousands of dimensions. 

Really hard to wrap your head around, but there could be hundreds of thousands of them. And each dimension. Captures a different aspect of the data's of meeting. So you could really get in there and have lots of means that's there. And the purpose of vectorizing. when you're creating these embeddings. is to be able to compare these vectors. Based on their proximity in this high dimensional space. vectors that are close together. Represent similar content. Even if using different words. 

 As we talk more about semantic search, as you think about what we talked about in the quick bites, talking about search and having. Natural language search or semantic search. The way that works. As you have less data. In Vectren, beddings, it's been vectorized. If you will. 

And now you're going to do calculations, and this is all happening with the vector databases. It's not a calculation that you would need to compute by hand. Or an engineer, your team. But the way that databases work as they're doing this. Proximity calculation. To return back to you. Vectors that are close together. 

And why this really matters. 

if we just think about the search aspect. 

We're able to go from exact matches. To semantic similarity. 

Okay. So traditional keyword search. Relies on the exact word matches. Or predefined synonyms. 

That works really well for finding information with known keywords. But it falls really short when the meaning is more important than the words. When I gave the example, of. Spotify finding podcast. You're able to look for a sentence. Even if we had another example of searching for. Electric vehicles. That may miss results that only mentioned cars. We talked about keyword and exact matching. 

You're thinking about. An index into a book. you have to have an index entry to find where that term is used on a page. So if you don't have those synonyms, you're going to miss the results. 

The vector search is going to capture the meaning. So if we had the vector search and search for electric vehicles. Cars is a similar concept, right? It's type of vehicle. The vector that represents cars, the vector that represents vehicles are going to have some overlapping concepts. So even though. It's not the exact same words. 

They're relevant based on meaning. So you would get back a search result that has that. They allow data points. With similar meanings. To cluster in space. 

For example , we talked about. Laptops a moment. I go, and I had mentioned about having  lightweight laptop for travelers. We may have a cluster. That's the laptops, tablets, smartphones. There might be another cluster for electric vehicles, bicycle scooters. And then when a query is made. 

The database is going to find the closest cluster. And retrieve the relevant content. This makes search much more intuitive for users. 

  Let's look at a really simple analogy.   Think of a city map. Whatever city you're in, your favorite city? Let's think about a city map. And I think about every location has GPS coordinates. 

Now, instead of looking for a specific address, that would be keyword search. 

You could instead do a search to say, find the closest at Thai and restaurant. 

Have you ever done that on Google maps or apple maps or ways or whatever? You're mapping. 

Tool of choice is. . I don't know how many times I've been going somewhere. And you want to find a gas station on your route. You want to find a restaurant? Coffee on your route. 

But similar to find the closest Italian restaurant. To me, find me the closest coffee in the morning. I find a gas station. That's close enough. 

So the search behind that. It doesn't need the exact name. It understands a category. Of what you're looking for. 

And then obviously has a GPS coordinates. And returns. So how this relates to vector search  in fact, our databases don't need exact keyword matches. They understand the context of your query. And they could return the closest matches. Just like the map gives you relevant restaurants without needing their exact names. 

Now imagine the Italian restaurants, GPS coordinates. Are changing dynamically. Based on popularity. Ratings. Menu changes. 

He vector databases are similar to that as well that they could update the vector representation over time. As new data or relationships emerge. 

Which means that the shirt's results stay relevant. Yeah. So if you were to get back, you imagined get back your search for your time restaurant. Close to the 10 restaurants to me. And if the search results are ranked by popularity. By ratings, whether they're open or closed. I had a data is always changing, especially popularity and ratings. Or menu changes. In fact, your database can have the same thing. 

You could be updating data. So the search results are always relevant, so it's not faxed. 

And if we look closer at how vector databases really do differ from this traditional databases. Again, traditional keyword search yet exact matching. They find results only if the search terms. Match awards and that document. Again, searching for laptops for travel. Only returns documents contain those exact words. Or synonyms if they were explicitly indexed by, the engineers working on it. You can imagine quite a few limitations with this. Requires manual curation as synonyms and related terms to improve accuracy. There's tools to help with that. 

But you miss kind of nuances that may not understand the meaning of the search term beyond just the words used. No CVV had synonyms for a laptop and send them's for travel. You're still matching on these words. Not the meaning of the words. 

 You could face challenges that others more cinnamons or added. The search performance can degrade that you know, is up for debate in some cases of how it's implemented and those types of things. So let's not concentrate as much on that, but realize that. The key limitation. Is. It's just looking for words, not meaning just like an index in a book. 

The vector search .

Instead of looking for these exact matches. They retrieved data points. That are conceptually similar to the query. 

 If I'm doing a search for  electric cars. This career, turn documents on battery technology. Or environmental impact. Even if those results and those documents that come back. Don't use the word electric cars. She may not see those words anywhere in those documents. But they are conceptually related. 

Big advantages over keyword search. 

Is this contextual understanding? 

You'll have much fewer false negatives. which is the case where users are less likely to see. No results. Since related contents retrieved. Even if the exact words aren't present. 

And this could be updated with new data. To keep results relevant over time without any sort of manual intervention. 

You can certainly keep a keyword search. Updated as well. So I don't know how much. You would need to really worry about that. But if you're concerned about a vector database being stale, It has the same capability that you could continuously be updating it. And have relevancy stay there. 

When you think about the business use cases and compare them. 

If we're doing traditional keyword. Customer searches for business travel laptop. 

And the database that you're using. Returns laptops with business travel and the product title or description. No product contains those exact words. The result may be empty or irrelevant. 

The vector search. 

And customer searches for business travel laptop. 

And the database understands that relates to lightweight portable laptops and returns, relevant products. Even if the phrase isn't present. And this obviously improves. The experience and reduces frustration. 

Walmart Is a business use case. 

Even if you don't shop at Walmart online. 

They've been adopting. Jareth AI in their search. It's particularly groundbreaking for what they've been able to do. 

So we talked about. Doing a search with a vector database. And having it return. 

Results that are conception unrelated. 

What Walmart has been doing. Is, you could do a search. Four. Let's say football watch party. And not only will it turn just snacks and chips. But also party drinks. Superbowl apparel. Televisions. So it really understands the nuanced needs of their customers. 

And  understands the intent behind the customer's query. Not only does this increase and enhance the shopping experience. 

But also significantly improves the efficiency of search operations. By storing data as a vectors. 

And a factor, just being a long list of numbers. That provides these systems, the ability to understand meaning and context. Rather than just key words. 

So this could really transform how businesses handle search. An AI powered interactions. 

Could really improve your customer satisfaction, operational efficiency. So that encourage you to rethink. Your search infrastructure. Could semantic search or AI powered chat using a vector database. Enhance customer or employee experiences. Are you already using this today? 

If you are, I would encourage you to spend some time with engineering teams that. Have built this. To really go beyond just our conversation and really understand how you're using it. Give you a lot of ideas of how could you continue to use this? How could you grow? This capability. 

So let's go a little deeper now that we've talked about. Vector databases and. The high level that there's an embedding that we're converting data structured, unstructured data. Into embeddings. And the database is going to do. A proximity search. Across a highly dimensional graph. 

To find things that are similar to return that back and how the vector database. Helps and really understand the meaning of the words and context. 

Let's talk a little bit about how data is actually converted to vectors in the first place. 

So these are called embedding models. 

One is Bert. 

Another is called word to vac. So you can think about that as words to vector. And open AI has their embeddings, which are extremely popular. So again, these embedding models. Translate. 

She when readable content, product description. Into high dimension vectors. 

And not just the content of, in the description, but again, the meaning behind it. And high dimensional vector is just a fancy way of saying a really long list of numbers. 

Again. So if we had a product description of a lightweight, portable laptop, That might map to a vector that places at close to other travel friendly devices. Even if those products don't use the same words. 

So for business, this really matters because these vector embeddings. They allow companies to surface related information. That keyword search would really mess. 

If you look at recommendations, this could ensure that you have raw, relevant recommendations. 

Even when users use varied phrasing. 

How is this done? 

So the vector search is performed. There's a variety of algorithms. The first. Is K nearest neighbor. 

You'll see that abbreviated often as K N N. 

And this. Search process of this algorithm. It's looking for the closest vectors to a query. Vector space. It's looking for the closest vectors. 

In that vector space. Very much like finding the closest restaurants on a map. 

this ensures that the results match. The intent, if you will, or the concept of the query. Not just the words. So there's a business use case,  if you had a search. And a customer searches for eco-friendly vehicles. Vector shirts can return products about electric cars. Or even hybrid bikes. 

But conceptually. 

They're related to eco-friendly vehicles. 

Another very popular algorithm is approximate, nearest neighbors. 

With this, in a large dataset, searching through every vector. It could be slow. 

When we think about having these very long lists of numbers. For representing your product descriptions for representing images, for representing audio, representing all these different types of data that you may have. It can become massive in size and it can be slow to find the vectors that match. So approximate, nearest neighbors. 

Algorithm is used to speed up searches. It could provide nearly accurate results. 

The trade-off is you lose some precision, a sacrificed for that performance. 

that's where it's you think about its approximate? So you lose a little bit of precision. For faster performance. But this can work really well for real-time recommendations. Where you really do have a limited amount of time to return a result to a user. And that performance envelope is really small. 

Now, when you think about the types of vector databases. 

There's a lot of vendors on the market. You'll see open source solutions. From Middleville from Weaver.  from quadrant. You'll see managed solutions from pine cone All the hyperscalers, whether it's Azure, AWS, Google cloud, everyone has them. So there's a whole lot of movement in this space and a lot of different vendors. 



The Forrester wave report that I mentioned. may not be a bad place to go, just to get an idea of the different vendors. 

Not suggesting that. You use that to make your decision? Not at all, not affiliate with Forester. But it gives you a, maybe a good idea as a starting point. Or a place to consider. 

 Vector databases  really Excel  in integrating with large language models. And serving as a memory for AI. 

These different. Large language models like chat CPT. Our state last, they don't remember your conversations. Once the session ends. And vector databases solve this problem. , they could be used to store past interactions. Or domain specific data. As vectors. 

As an example, a customer service chat bot. Can use a vector database to retrieve past conversations that a customer had with the chat bot. 

It could retrieve past conversations that were had over the phone. With customer support or via email. If you think about all the interactions that a customer had with your business. You could store all of that. And all that conversation. And now you're able to provide a much more personalized interaction over multiple sessions. Whereas the chat bot, if it's based on a large language model, Every time, there's a new session. 

it has no memory of that because it doesn't they're stateless from that standpoint. Once the session's over. It's completely stateless. 

If you think about. Domain specific content, not just extending its memory or providing it a memory. But domain specific content. Now you could use this, the vector database that is. To provide specific business data. Like your product catalog or policies. These can be provided as a vector embeddings. Now you could give the LLM. Power chat bot or assistant much better context for user queries. 

an example, legal firm can store case law summaries as vectors. And when employees ask questions. Of the AI assistant, the assistant can retrieve the most relevant case summaries. 

And provide those back. 

a really good example of this is Morgan Stanley. 

In 2023. They debuted a new tool for employees. Is an AI assistant. That's able to answer common investing. And personal finance queries. 

This is using over a hundred thousand research documents. I'm sure it's much larger Corpus of documents now than it was then. 

So this allowed. Employees. To very quickly. Have an extremely fast documents search. In a chat type of experience. So this is a. Search and a chat interface to allow them to interact. With all these documents. So really powerful. 

This is not without its challenges, though. All right. So we've talked about. Vector databases from a high level, if you will, or a little bit deeper. As to really what they are and why they're there and how to use them, or how to think about them. We've talked about some of the embeddings. And how were. Creating. Embeddings and vectors of this data. All right. 

Remember? Just think about. A vector taking a word and converting it to numbers. Not just a word to numbers. But. A word. Two context and intent. So taking a sentence and getting the context. And. Intent from it. But there's challenges to be aware of. you may need to have people on your team that have knowledge of how to build them to numb. Managing the embeddings, make sure the vector queries are effective. It could be hard to integrate with legacy systems, but there's a lot of tools in this space. 

There's a lot of things changing very quickly. So some of that integration. And the struggles with it'll get easier and easier. 

 Searching high dimensional vector spaces. Again, it's expensive computationally. So you'll see different techniques. You probably hear about sharding. You may be familiar with that term from databases that you've been around a new past. Or approximate, nearest neighbor. Good help manage performance with large data sets. 

The vendors in this space, continue to keep pushing, to make things more performance as well. So some of that will Start to probably become less of a concern. But it's always good to verify, to do testing of whatever system you're thinking about. And the data that you're planning on using. 

So as you think about this it asks you several questions, right? Or your current search tools, keyword based. 

If they are, would switching to semantic search, improve your user experience or employee experience. Are there quick wins that you can identify? Is there an internal knowledge management solution. That you could deploy semantic search for faster employee access to documents and tools. Are you using customer service chatbots? Again, here you can improve personalization. By giving the chatbots memory of past interactions. 

So if you're using chat today, Can you enhance it? 

I would, again, I'd explore the vendors, whether it's opensource, managed solutions. Really make sure that it's one that fits your organization's needs. Are you using vector search today? And if not ask him how we can start. 

So in summary today, we've gone through. the tour. Of what are the vector databases? From what is it? Why. We went a little bit deeper and understanding embeddings, understanding that their index or shirts at happens a couple of algorithms. 

These technologies again. Fantastic for enabling natural language search. And AI memory. And they can enhance both customer experience and internal operations. There's a lot of different use cases you could use. 

To start to think about where can you apply semantic search. Or memory augmented AI in your business. 

I'd encourage you to subscribe and. Listen to the next episodes of this podcast,  and please, I invite you to share your feedback. Ask questions, leave comments below. Or on social platforms. Or feel free to reach out to me directly if you'd like to.