#7 - RAG: Bridging AI and Business Data for Executives - Quick Bytes

In this podcast episode, host Andrew Psaltis of Dragonfly Risings School of Data discusses how executives can leverage Retrieval Augmented Generation (RAG) to make AI systems business ready. Highlighting the benefits of RAG and practical applications to help you understand how you can apply this to your business.

Transcript

 Dragonfly Risings School of Data proudly presents the AI demystified for executives podcast. This is the podcast for executives who want to learn how to apply aI to their business. I'm your host. Andrew Psaltis 

This week's topic is rag retrieval, augmented generation. Making AI systems business ready? 

A common concern that you may have. And a question is how do we make AI use our business data? So we've talked in previous episodes about the background of GPT. A little bit on large language models. Some of the historical context 

And I'm sure you've heard about how these are all trained on the Corpus of the internet. But they're not trained on your business data. 

Retrieval augmented generation rag for short, and we'll just call it rag. Gives you an opportunity to actually have the model, use your business data.  It's not going to use your data for training, but using your data for being able to make decisions. So let's briefly walk through in the time we have today of how this works. And why this matters for you. 

When we talked in previous episodes about vector databases. And we talked about how they're used for natural language search, which is a fantastic way of using them. One of the other key things we talked about. Was giving.  Large language models or other AI systems, their memory. This is an opportunity where you use your business data. If you will, as this memory and as, as context. 

Let's take an example. That you are providing. A. Chat application for your field support team in your field engineers, to be able to repair equipment. In the field. So you may have hundreds. Of different product documents, all these different PDFs. And you want to provide them the ability to have a chat GPT leg experience. But using your documents. So the way that you make this work, and this is where RAC comes into play is. As I'm a technician in the field. And I entered into a  text box. 

I'm trying to understand. What to do with a power failure of this piece of equipment. 

When they enter that the next thing that you do from the software standpoint. Is you go and retrieve the documents from a vector database. That is related to what they're asking. From there you take those documents. this is now called the context. you give that context along with the user's prompt. Chu the model. GPT  or another model. You give it. The prompt that the technician typed in. Along with the content. And ask it to provide the results based upon all of this content you just gave it. 

It'll then use its language and reasoning skills to look at the documents that you provided as the context. along with the prompt. And craft the response. 

Okay. So you could think about this as a hop in the middle, if you will. So it's the interaction from the end user with your application. Where this AI system. You then get the most relevant information from a vector database. that's the retrieval augmented. You pass that information to the model. To have it's due it's generation. 

And then it goes back to the user. So we're getting the model. To use your business data, that it was never trained on. And it won't be trained on the data you pass into it. But you're providing it your data. On the fly. As a user interacts. You may be thinking there's another option I've heard about, to do fine tuning and with fine tuning. I could do the same thing. 

I could give the model, my data. And with that, I could end up with my fine tune model that is now been enhanced with my business data. There may be cost differences could be significant between fine tuning and rag. There's also the latency of having to tune the model. And you could think about the case of if your system is multi-tenant and Servicing multiple types of customers. Multiple use cases. You may not want to have a fine to model for every single one. All right. 

So rag allows you to pull out your data if you will, and keep that separate. And be able to let your data, whether that is, customer support, technical documents, whatever that may be. Your domain specific data, keep that separate and then pass it in. In real time to the model. 

Another way that you could use this. As perhaps we weren't doing this for supporting fueled engineers. But you want to have all the chat? Conversations that a customer had with your AI system. You could store all of the chat conversations. All the transcripts could be stored in this vector database, which is what rag uses. Store it there. And now you use that context. With the model.  

So if you think about this, of how do you make these AI systems business ready? By really using. Your domain data. Rag is a way that you do that. So again, retrieval augmented generation. We're doing retrieval first. To augment. The model. And then it does its generation. So think about you type into a text box, whatever that may be or some way you're interacting with the system. 

And you're going to grab your domain data. You're going to pass that. To the model. And that's called its context. And you're going to ask the model two. Do its magic. If you will. Quote, unquote, I to use his language generation skills and reasoning. With the prompt passed in this other context. 

A nice thing about this. Besides just the fact that it is. Using your data. Too. Generate a good response is this helps eliminate hallucinations as well. Because now we're helping ground the model. And give it. The source of truth and the data that it's to operate with. So you could specify. To the model to use this prompt. And only use the data provided as context. When it is. Creating its response. We could. Have the model use your domain data. And really help the systems. Become business ready? 

That's our quick bite for this high-level overview of rag. Tune into our next episode. We're going to go into this in a bit more detail and pull back the covers a little bit more. So you have a deeper understanding as to how this works. Some of the options that are there. And help you further realize how you may be able to apply this to your business. 

 Please remember to subscribe. To her, every listen to his podcast. So you stay up to date. It never hesitate to reach out, ask any questions that you may have.

Thanks. That's all for now.