Python course Β· Module 11: RAG and Multi-Agent Systems
RAG - Extended Memory for AI
In this lesson6
Imagine a guide who knows the savanna only from books printed three years ago. You ask about a new trail to the watering hole, and he confidently describes a path that no longer exists. That is how a language model behaves without access to current knowledge. Welcome to the penultimate stage of Python Safari! In this location we will teach the model to check its field notebook before it says anything.
Just as large mammals evolved bigger brains and better memory, language models evolve thanks to RAG (Retrieval-Augmented Generation). It is a technique that gives AI access to external knowledge without retraining the model.
What you will learn
- recognise the LLM limitations that RAG solves,
- describe the three pipeline steps: retrieve, augment, generate,
- build the simplest RAG in plain Python with OpenAI and NumPy,
- split long documents into pieces (chunking),
- measure retrieval quality with precision and recall.
The Problem with Traditional LLMs
Language models have fundamental limitations, much like animals adapted only to a single environment:
Limitations of Base LLMs:
- Knowledge cutoff - knowledge frozen at training time
- Hallucinations - generating false information
- Lack of company context - they don't know your documents
- High fine-tuning costs - adapting the model is expensive
The first limitation matters most: the model knows nothing about what happened after training ended, and it has never seen your private documents. RAG does not change the model itself. It changes what the model gets to read before answering.
What is RAG?
RAG is an architecture that combines information retrieval with text generation. The diagram below follows a question from the user all the way to the final answer:
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β RAG Pipeline β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ€
β β
β User question β
β β β
β βΌ β
β βββββββββββββββββββ β
β β Embedding β β Convert to vector β
β ββββββββββ¬βββββββββ β
β β β
β βΌ β
β βββββββββββββββββββ βββββββββββββββββββ β
β β Vector Search ββββββΆβ Vector Database β β
β ββββββββββ¬βββββββββ βββββββββββββββββββ β
β β β
β βΌ β
β βββββββββββββββββββ β
β β Retrieved Docs β β Top-K similar documents β
β ββββββββββ¬βββββββββ β
β β β
β βΌ β
β βββββββββββββββββββ β
β β LLM + Context β β Generate with context β
β ββββββββββ¬βββββββββ β
β β β
β βΌ β
β Answer β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββRead it from the top. We turn the question into a vector of numbers (an embedding), search a vector database for passages with a similar meaning, take the best few (Top-K) and paste them into the prompt. Only then does the LLM generate the answer. The model does not change, only its "cheat sheet" for this one question does.
Basic RAG Implementation
Let's build the whole pipeline in a few dozen lines. We start with the OpenAI client, NumPy and a tiny knowledge base, a list of sentences that plays the role of our field notebook:
from openai import OpenAI
import numpy as np
client = OpenAI()
# Simple knowledge base
knowledge_base = [
"Python Safari is a Python programming course with 12 modules.",
"RAG combines document retrieval with text generation by LLMs.",
"Embeddings are vector representations of text in semantic space.",
"Vector databases store and search embeddings efficiently.",
"LlamaIndex is a framework for building RAG applications.",
"CrewAI enables the creation of multi-agent systems."
]OpenAI() reads the API key from the OPENAI_API_KEY environment variable, so we never type it into the code. The knowledge base itself is an ordinary list of strings, nothing magical.
Now two helper functions. The first sends text to the text-embedding-3-small model and returns a vector, the second computes cosine similarity, which tells how much two vectors "point" in the same direction:
def get_embedding(text: str) -> list[float]:
"""Generates an embedding for the text."""
response = client.embeddings.create(
model="text-embedding-3-small",
input=text
)
return response.data[0].embedding
def cosine_similarity(vec1: list, vec2: list) -> float:
"""Computes cosine similarity of two vectors."""
vec1, vec2 = np.array(vec1), np.array(vec2)
return np.dot(vec1, vec2) / (np.linalg.norm(vec1) * np.linalg.norm(vec2))Cosine similarity ranges from -1 to 1, and the closer it gets to 1, the closer the meanings of the texts. We will take it apart in the next lesson.
The third step is the search function. For every document we compute its similarity to the question, sort in descending order and return the best top_k:
def retrieve_relevant_docs(query: str, docs: list, top_k: int = 3) -> list[str]:
"""Retrieves the most similar documents."""
query_embedding = get_embedding(query)
similarities = []
for doc in docs:
doc_embedding = get_embedding(doc)
similarity = cosine_similarity(query_embedding, doc_embedding)
similarities.append((doc, similarity))
# Sort by similarity
similarities.sort(key=lambda x: x[1], reverse=True)
return [doc for doc, _ in similarities[:top_k]]Notice one weakness: this function computes the embedding of every document on every question. With six sentences it does not hurt, with thousands of documents it would be slow and expensive. That is why in later lessons we compute document embeddings once and keep them in a vector database.
Finally we assemble everything in rag_query, where you can see the three letters of the acronym: Retrieve, Augment, Generate:
def rag_query(question: str) -> str:
"""Full RAG pipeline."""
# 1. Retrieve - find relevant documents
relevant_docs = retrieve_relevant_docs(question, knowledge_base)
# 2. Augment - build prompt with context
context = "\n".join(relevant_docs)
prompt = f"""Answer the question using ONLY the context below.
If you cannot answer based on the context, say "I don't know".
Context:
{context}
Question: {question}
"""
# 3. Generate - produce the answer
response = client.chat.completions.create(
model="gpt-4o-mini",
messages=[{"role": "user", "content": prompt}]
)
return response.choices[0].message.content
# Test
answer = rag_query("What is RAG?")
print(answer)The key part is the instruction in the prompt: "using ONLY the context below", plus permission to answer "I don't know". That is what limits hallucinations. The gpt-4o-mini model is exactly the same as a moment ago, only the prompt changed.
Chunking - Splitting Documents
Long documents need to be split into smaller pieces (chunks), because an embedding of a whole book blurs the meaning and a prompt has a limited length. The simplest method cuts the text every fixed number of characters, with an overlap:
from typing import Generator
def simple_chunker(text: str, chunk_size: int = 500, overlap: int = 100) -> Generator[str, None, None]:
"""Simple function for splitting text into chunks."""
start = 0
while start < len(text):
end = start + chunk_size
chunk = text[start:end]
yield chunk
start = end - overlap # Overlap to preserve contextThe overlap makes sure a sentence cut at a boundary lands in full in at least one chunk. Just keep overlap smaller than chunk_size, otherwise the loop never ends. The last chunk can also be very short, because it is only the tail of the text.
Splitting by sentences preserves meaning better. A regular expression cuts the text after a full stop, exclamation mark or question mark, and then we glue a few sentences together:
def sentence_chunker(text: str, sentences_per_chunk: int = 5) -> list[str]:
"""Splits text into chunks by sentences."""
import re
# Simple sentence segmentation
sentences = re.split(r'(?<=[.!?])\s+', text)
chunks = []
for i in range(0, len(sentences), sentences_per_chunk):
chunk = ' '.join(sentences[i:i + sentences_per_chunk])
chunks.append(chunk)
return chunks
# Usage example
document = """
Python is a high-level programming language. It was created by
Guido van Rossum. It is very popular in Data Science and Machine Learning.
RAG is a technique combining retrieval with generation. It allows LLMs
to use external knowledge sources. It is crucial for enterprise AI.
"""
chunks = sentence_chunker(document, sentences_per_chunk=2)
for i, chunk in enumerate(chunks):
print(f"Chunk {i}: {chunk[:50]}...")Every printed chunk now contains whole sentences, and the document content has not changed, only its packaging has. I recommend starting with sentence splitting, because chunks cut in the middle of a thought produce weaker embeddings.
RAG Quality Metrics
How do you check whether RAG works well? We evaluate two stages separately: did retrieval find the right passages, and does the answer stick to the context. Here is a class skeleton with three typical metrics:
from dataclasses import dataclass
@dataclass
class RAGMetrics:
"""Metrics for evaluating a RAG system."""
def relevance_score(self, query: str, retrieved_doc: str) -> float:
"""Is the document relevant to the question?"""
# In practice we use an LLM for evaluation
pass
def faithfulness_score(self, answer: str, context: str) -> float:
"""Is the answer grounded in the context?"""
# Checks for hallucinations
pass
def answer_correctness(self, answer: str, ground_truth: str) -> float:
"""Is the answer correct?"""
# Comparison with ground truth
passThe methods contain only pass, because in practice relevance, faithfulness and answer correctness are usually judged by a second model acting as a referee. It is a skeleton to fill in, not a finished tool.
Retrieval, on the other hand, can be measured with plain set arithmetic. Precision tells what share of the retrieved documents is relevant, recall tells what share of the relevant documents we found:
# Simple evaluation example
def evaluate_retrieval(query: str, retrieved: list[str], relevant: list[str]) -> dict:
"""Computes precision and recall for retrieval."""
retrieved_set = set(retrieved)
relevant_set = set(relevant)
true_positives = len(retrieved_set & relevant_set)
precision = true_positives / len(retrieved_set) if retrieved_set else 0
recall = true_positives / len(relevant_set) if relevant_set else 0
f1 = 2 * precision * recall / (precision + recall) if (precision + recall) > 0 else 0
return {
"precision": precision,
"recall": recall,
"f1": f1
}Example: if the system returned ["a", "b", "c"] and the relevant ones were ["a", "d"], precision is 1/3, recall is 1/2, and F1, the harmonic mean of both, is 0.4.
RAG is the foundation of modern enterprise AI applications. In the next lesson you will learn about embeddings - the heart of the entire search system - and see why similar meanings end up close to each other.
Remember from this expedition: RAG is a guide who checks the field notebook before every answer.
Docs entries used in this lesson
Spotted a mistake in this lesson?
Check yourself
Answer the questions from this lesson. Pick an answer to see right away whether it is correct.
1. What does the acronym RAG stand for in the context of AI?
2. Which limitation of traditional LLMs does RAG help solve?
Hands-on tasks in the game
- Code editor
Implement a basic RAG pipeline
- Vertical ordering
Arrange the steps in order:
- Click in order
Arrange the RAG pipeline steps:
- Vertical ordering
Arrange the RAG query pipeline steps in the correct order: