In today’s AI hype you cannot miss the term “RAG,” which stands for Retrieval Augmented Generation. In plain English, it stands for customizing large language model reasoning with your own context and knowledge. I searched a lot of resources and AI-generated content for this fairly simple technique to be explained well. I’m still looking for the perfect article in that sense! Hopefully this thread could save you a bit of time in grasping the essence of RAG.
This is the first part of a blog post series, where a few theoretical chapters are explained before we start getting our hands dirty with RAG-related code! So please bear with me!
Foundation Model
Foundation models are those large language models that are fairly computationally heavy to train (usually simple everyday users don’t have the computational power or the vast dataset to train these models), and it takes a lot of time to train them. They have a cut-off date well back in the past, due to the previously mentioned reasons. It’s also characteristic of these models that they are general-purpose, so you can achieve a variety of tasks with them: recognize an image, parse a PDF file, generate content, etc. Some examples of foundation models are Claude’s 3.5 Sonnet, GPT-5, etc. Of course, you are not restricted to closed-source models; you can also reach significant results with open-source models like DeepSeek or LLaMA. But that requires some further tinkering in the software architecture. As a fan of open-source software and self-hosting, this is going to be part of the code example.
Why should I use RAG in the first place?
Everybody wants to incorporate AI appliances into their products, but one thing stands out: these LLMs are not specialized in your area of expertise (since they were trained on the entirety of the internet’s content).
Imagine that you have a self-managing support chatbot on your user interface with the knowledge of your entire user manual. Or you require tailor-made executive summaries based on many articles, blog posts, videos, diagrams found on the internet in various media channels, just to name a few use cases.
It would be beneficial to control or customize how the LLM responds in certain cases to serve your purpose. The RAG technique tries to extend an LLM’s capabilities, for example:
- Represent up-to-date information (which LLMs struggle with because of the lengthy training process and cut-off dates).
- Have more specialized or self-tailored knowledge on certain topics than the basic LLM.
- Use citations or source attributions to enhance user trust, since everybody (including me) is skeptical. This way, users can verify the sources themselves.
How can I implement a RAG solution?
To have a “customized” generative AI functionality in your web application, you have to intercept the calls made to the LLM and enrich them with further context, to get more accurate, tailor-made, and trustworthy answers. But how do you provide the relevant context based on the initial inquiry? That’s where vector databases and similarity search come in! But before we jump into all that, I would like to introduce you to the definitions used in the previous sentence!
Vectors
If you paid a little bit of attention in high school mathematics lessons, vectors should ring a bell. The vector construct could be defined as the following: an interval that has its length and associated direction. Take the following example:
In a two-dimensional coordinate representation, the vector can be described by a line drawn to the point P [1,1] from the origin point of the coordinate system, representing its direction and length.
Vector Distance
Between two vectors you can define distance, which can be measured between the vectors’ endpoints. In the illustration, you can see the distance between vectors “u” and “v” with the dotted line.
Vector Databases
Now that we are familiar with how vectors can be constructed and their distance calculated in two dimensions, let’s imagine that these concepts work not just in two dimensions, but in as many dimensions as we like, and we are able to calculate the distances between those multidimensional vectors as well! We could represent a single entity as a multidimensional vector (for example, a cat), based on different traits/features considered as the dimensions (for example, height, weight, life expectancy, etc.), as shown in the illustration.
Vector databases, for example PostgreSQL’s pgvector extension, can efficiently store and search these multidimensional vectors. The search is performed by having an input multidimensional vector, calculating the distances to nearby vectors, and retrieving the closest hits to the input vector.
Okay, but how do we manage to create these vectors from a single entity, or token? That’s what embedding models are for.
Embedding Model
These models are more specialized than foundation models. They are capable of understanding relationships and meanings between the input words, images, tokens, and can transform them into a numerical representation that machines can understand: vectors.
For example, “cat” and “kitten” would have a similar vector representation, indicating these tokens are closely related to each other. Embeddings are the essential building blocks for our personalized vector database content, but are also crucial for large language models as well.
Now we understand all the necessary building blocks to enrich the context for the LLM!
Example Use Case
Now that we understand all the necessary building blocks to enrich the context for the LLM in our web app, let’s walk through a simple use case end to end. Please make sure that you understand the theoretical content above!
Let’s imagine that we would like to create a knowledge base from our company’s internal documentation archive, and ask the LLM questions about it. In the attached image you can see the necessary steps to assemble a RAG solution.
Data Preparation
In the first step, you have to acquire the data from your documentation archive (Confluence, Headless CMS, etc.). I’ll let you deal with that yourself.
To stay relevant in context, it’s a good practice to split these documents into digestible chunks for the LLM. Chunking could save you money as well, since the token cost of the LLM would be reduced. These document chunks also have different features (for example, meaning); they are going to be our unit for vectorization.
After the chunks are built, the system lets the embedding model vectorize the chunks, and store the chunked document vectors into the vector database.
Your knowledge base has been fed with customized data!
Retrieval Augmented Generation
The vector database is filled with data; now it’s time to intercept the query initiated to your web application, before hitting the LLM’s API, and enrich the input query message with your personalized context.
Call the embedding model to vectorize the initial query, then run the vectorized query against your vector database’s content to find similar entries (3–4 entries would suffice, but that varies based on token limits and budget).
Attach the relevant document content and its metadata along with the initial query in the request body for the LLM API, and voilà! You’ve got your personalized GenAI answer! The illustration above shows such a business case, where it can tell about the month’s spending by looking at the individual’s related data.
Achieving this simplest RAG-enabled backend endpoint functionality can be completed with basic backend and Docker expertise. You can also cut down the expertise level by utilizing n8n workflows for the backend part, and using cloud services for the database and model dependencies, at the cost of flexibility.
Next Steps
In this lesson we delved into the devilish details of a RAG-based architecture, and how you could leverage LLM capabilities and personalize them in your favor!
In the next blog post episode, we will dive deeper into an implementation of a RAG solution with TypeScript. Stay tuned!
