Chatbots - Conversational AI

Evening in camp, you ask your script two questions back to back. First: "I am on safari in Kenya." Then: "Which animals will I see here?" The model answers politely and at length, except it has no idea where "here" is - it lists pandas, tigers and polar bears in a single paragraph. Nothing broke, you did not lose your API key, you did not misspell the model name. That is simply how it works, and until you understand why, every chatbot you build will have the memory of a mayfly.

In this lesson you will build a Darwin who can actually hold a conversation. First by hand, on a plain Python list, so you see the whole mechanism with no magic in it. Then the same thing again with LangChain's ready-made parts. Finally you will give him knowledge the model does not have - the text of your reserve's field guide - and expose the whole thing as a service the entire research station can call.

A model that remembers nothing

Let us start by watching the problem happen. Below you send two independent requests to the same model, in the same script, one after the other. The first sentence says where you are, the second asks a question that makes no sense without that information. Pay attention to what exactly lands in the

messages

field on the second call: one single sentence, with no trace of the previous one.

client

is the client object you know from earlier lessons, and

chat.completions.create

is the same call path you have already walked.

1from openai import OpenAI
2
3client = OpenAI(api_key="your-api-key")
4
5first = client.chat.completions.create(
6    model="gpt-5",
7    messages=[{"role": "user", "content": "I am on safari in Kenya."}]
8)
9
10second = client.chat.completions.create(
11    model="gpt-5",
12    messages=[{"role": "user", "content": "Which animals will I see here?"}]
13)

Let us count what did not change between those two calls: the client is the same, the key is the same, the model is the same, the Python process is the same, and the variable

first

is still sitting in your program's memory. Exactly nothing changed - and that is the whole point. The server that handled the second request has no idea the first one ever existed. The chat API is stateless: every request is an independent parcel, and the model sees only what you pack inside it.

That is why "the model remembers all conversations by itself" is false. A model is a frozen set of weights - it stores nothing once the answer is finished, and it has nowhere to store it. "It is not possible" is equally false, because the rest of this lesson is one long demonstration that it very much is. The real answer is far less spectacular than people expect: a chatbot's memory is message history that you keep on your side and resend together with every new request.

Memory is just a list

Since the model only ever sees the contents of the

messages

field, all you have to do is keep the minutes of the conversation yourself and send them in full every time. Those minutes are a list of dictionaries, and each dictionary has two keys:

"role"

says who spoke,

"content"

carries what was said. There are three roles.

"system"

is the instruction for the model - who it should be and how it should behave - and it always sits at the very front of the list.

"user"

holds your messages.

"assistant"

holds the model's replies. Let us start with the system instruction alone, before the first question is even asked.

1messages = [
2    {"role": "system", "content": "You are Darwin - a friendly safari guide."}
3]

That is your chatbot's entire memory at the moment of startup: an ordinary Python list with one dictionary inside it. No database, no library, no state on OpenAI's servers. The system instruction goes to position zero and stays there until the conversation ends - it is the one message you never remove, because it is what keeps Darwin in the role of a guide instead of letting him drift back into being a generic assistant.

Now you append the user's question. The method for adding an item to the end of a list is

.append(

, and what you put inside it is a new dictionary with the

"user"

role. The key order is always written the same way:

"role"

first, then

"content"

. The question itself lives in the variable

text

, so under the

"content"

key you put that variable, with no quotation marks around it - it is a variable, not a string literal.

1text = "I am on safari in Kenya."
2
3messages.append({"role": "user", "content": text})
4
5print(len(messages))
6print(messages[-1])

Run it and you will see

2

and then

{'role': 'user', 'content': 'I am on safari in Kenya.'}

- the list grew by one item, and

[-1]

points at the last of them. Let us take that single line apart, because you will be writing it hundreds of times:

messages

(the name of the list),

.append(

(the method that adds to the end),

{"role": "user", 

(opening the dictionary and the role),

"content": text}

(the content and closing the dictionary),

)

(closing the method call). In exactly that order, because the dictionary has to close before the

append

parenthesis does.

It is worth noticing what that operation did not do. It sent nothing over the network, it cost you not a single token, and it told the model nothing at all. The only thing that changed was the contents of a variable inside your own program. Which quietly rules out another tempting answer: browser cookies. A cookie lives in the user's browser and never reaches the model - at best it can carry a session identifier, but the minutes of the conversation still have to sit on the server side and still have to be attached to the request. The HTTP channel has no concept of "the previous message".

Now we close a full turn of the conversation inside a function. It appends the question to the list, sends the whole list to the model, pulls the answer text out along the path

answer.choices[0].message.content

that you already know, and then appends that answer back onto the list under the

"assistant"

role.

1def ask(text):
2    messages.append({"role": "user", "content": text})
3
4    answer = client.chat.completions.create(
5        model="gpt-5",
6        messages=messages
7    )
8
9    reply = answer.choices[0].message.content
10    messages.append({"role": "assistant", "content": reply})
11    return reply

The heart of this function is

messages=messages

: the entire history so far travels to the model, not just the current question. The second

append

matters just as much, and it is the one people forget. If you only recorded the questions, the model would never see its own words, and after a few turns it would start repeating answers or contradicting itself. The conversation has to be complete from both sides.

There is a price for this, and you may as well know about it now. On the twentieth turn you are sending nineteen previous exchanges to the model, you pay for every token in every request, and eventually the history stops fitting in the context window. That is why real chatbots trim the minutes: they keep the system instruction plus the last dozen or so messages, and summarise or drop the older ones. Memory is not free - it is simply yours.

ChatMessageHistory, or memory the LangChain way

A hand-rolled list is perfect for understanding the mechanism, but in a real project it quickly grows extra code around it: trimming, several conversations at once, format conversions. LangChain has a ready-made part for exactly this, called

ChatMessageHistory

. Before you import it, read the import line from left to right, because its order never changes:

from

→

langchain_community.chat_message_histories

→

import

→

ChatMessageHistory

. First

from

points at the module, then

import

names the thing you are pulling out of it. The module address itself means something too:

langchain_community

is the package of community-maintained integrations, and

chat_message_histories

is written in the plural because many kinds of history live in there - in memory, in a file, in a database. The class at the end is capitalized, the module is lowercase, and the two cannot swap places.

1from langchain_community.chat_message_histories import ChatMessageHistory
2
3history = ChatMessageHistory()
4history.add_user_message("I am on safari in Kenya.")
5history.add_ai_message("Great! Kenya is a paradise for animal watchers.")
6
7print(len(history.messages))

It prints

2

. This is the same list as before, only dressed up as an object:

add_user_message

corresponds to appending the

"user"

role,

add_ai_message

to the

"assistant"

role, and the

.messages

field gives you access to everything collected so far. Notice what is not here: no model call, no API key, no cost.

ChatMessageHistory

is a notebook, not a conversation partner - it holds the record and nothing beyond that.

Four steps to build a chatbot

The rest of the lesson is assembling a working bot, and it is worth memorising the order in which that is done. Step one: define the system prompt, that is, Darwin's identity. Step two: configure memory. Step three: create the chain that ties prompt, model and memory into one whole. Step four: implement the conversation loop that feeds the whole thing question after question. The order cannot be rearranged - the chain has nothing to tie together until the prompt and the memory exist, and the loop has nothing to call until the chain exists.

Step one.

ChatPromptTemplate.from_messages

builds a conversation template out of a list of elements, and

MessagesPlaceholder

is a special element marking the hole that the history will drop into. The

variable_name

parameter gives that hole a name. The last entry is the user's current question, inserted under the name

input

.

1from langchain_openai import ChatOpenAI
2from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder
3
4llm = ChatOpenAI(model="gpt-5", temperature=0.7)
5
6system = """You are Darwin - a friendly Safari guide.
7You answer enthusiastically and share fun facts about animals.
8You always look after the safety of the tourists."""
9
10prompt = ChatPromptTemplate.from_messages([
11    ("system", system),
12    MessagesPlaceholder(variable_name="history"),
13    ("human", "{input}")
14])

The template has three elements and each plays a different part. The system instruction is fixed and repeats on every turn - it is what makes Darwin enthusiastic and keeps him watching out for safety. The placeholder is empty on the first question and grows with every exchange after that. The pair

("human", "{input}")

is the slot for the current question. The name

"history"

is not arbitrary, and in a moment it will have to match the

history_messages_key

setting character for character - a mismatch in that one name is the most common reason a bot appears to work and yet remembers nothing.

Step two. Memory has to tell speakers apart, because several people will be using Darwin at once. For that you use an ordinary dictionary called

store

, where the key is a session identifier and the value is a separate

ChatMessageHistory

object. The function

get_session_history

takes an identifier and returns the history of that one conversation, creating it on first contact.

1store = {}
2
3def get_session_history(session_id: str):
4    if session_id not in store:
5        store[session_id] = ChatMessageHistory()
6    return store[session_id]

You will find a shorter variant online where a single global history is handed to everybody instead of a dictionary. I recommend the dictionary version, because the other one saves you two lines and buys you two very real problems: two people share one set of minutes, so Darwin answers tourist B based on what tourist A said, and on top of that you cannot clear one conversation without wiping them all. The same

session_id

always means the same conversation, a different

session_id

means a clean slate.

Step three. You know the

|

operator from the previous lesson - it glues the prompt to the model to form a chain. What is new is

RunnableWithMessageHistory

, a wrapper that injects the history into the placeholder before every call and appends both messages to the right session afterwards. The

input_messages_key

parameter tells it which input field is the question, and

history_messages_key

tells it which hole to pour the history into.

1from langchain_core.runnables.history import RunnableWithMessageHistory
2
3chain = prompt | llm
4
5chatbot = RunnableWithMessageHistory(
6    chain,
7    get_session_history,
8    input_messages_key="input",
9    history_messages_key="history"
10)

Notice that the chain

prompt | llm

itself did not change by a single character and still knows nothing about memory. The wrapper does not rewrite it and does not swap the model out - it only adds two moves around it: read the history before, write it after. You pass

get_session_history

to

RunnableWithMessageHistory

as a bare function name, with no parentheses, because it is LangChain that will call it at the right moment, handing it the identifier from the configuration.

Before you write the loop, let us test the bot on two questions. The session identifier is passed in a

config

dictionary, under the nested key

"configurable"

. The

invoke

method is the same one you use with any other chain - the only difference is that you hand it a

config

as well.

1config = {"configurable": {"session_id": "user_123"}}
2
3answer = chatbot.invoke({"input": "Hi! I am on safari!"}, config=config)
4print("Darwin:", answer.content)
5
6answer = chatbot.invoke({"input": "Which animals live here?"}, config=config)
7print("Darwin:", answer.content)

This is the same pair of questions we opened the lesson with, and this time the second one makes sense - Darwin knows that "here" means the safari mentioned in the first message. Not because the model got smarter, but because on the second call the whole history travelled to the server. Swap

"user_123"

for a different string and Darwin forgets the conversation on the spot, because he looks into a different pigeonhole of the

store

dictionary. You still read the answer through

.content

, exactly as with a plain chain.

Step four. The conversation loop is a simple console program: read a line from the user, check whether it is a quit command, send it to the bot, print the answer, repeat. The

input

function waits for text from the keyboard,

.strip()

trims stray spaces,

break

ends the loop, and

continue

jumps straight to the next pass without running the rest of the body.

1def run_chatbot():
2    print("Safari Chatbot - type quit to finish")
3
4    config = {"configurable": {"session_id": "cli_session"}}
5
6    while True:
7        question = input("You: ").strip()
8
9        if question.lower() in ["quit", "exit", "q"]:
10            print("See you on safari!")
11            break
12
13        if not question:
14            continue
15
16        answer = chatbot.invoke({"input": question}, config=config)
17        print("Darwin:", answer.content)
18
19if __name__ == "__main__":
20    run_chatbot()

The two conditions look alike and do completely different things. The first one ends the program, the second merely ignores empty input - if you put

break

there, one stray Enter would shut the whole conversation down. Writing

question.lower()

lets people leave with

quit

as well as

QUIT

. The session identifier is fixed here, because there is one person sitting at the console, and it lives outside the loop so that every pass lands in the same history. That is the full set: system prompt, memory, chain, conversation loop.

RAG, when Darwin has to know your field guide

Your bot talks beautifully about the savanna in general, but ask it about the safety rules that apply in your particular reserve and it will start inventing them. It has no way of knowing them - the camp rulebook was never in the training data. You can fine-tune a model of your own, which costs a fortune and has to be repeated every time the rules change, or you can reach for RAG, short for Retrieval Augmented Generation.

RAG is a technique for enriching a model's answers with information from your own documents: before you ask the question, you search those documents for the passages that match it best and paste them into the prompt as context. Let us clear up three misunderstandings right away. RAG is not a file format - the documents can be

.txt

,

.pdf

or

.md

and the technique does not care in the slightest. It is not a type of neural network - it changes neither the architecture nor the weights of the model, which stays exactly as it was the whole time. And it is not an API protocol - there is no endpoint and no communication standard by that name, and all the work happens inside your own code, before you send an entirely ordinary request.

The pipeline has five steps in a fixed order. Load the documents. Split them into chunks, that is, do the chunking. Create embeddings of those chunks. Search for the chunks most similar to the question. Generate the answer with the context you just gathered. The first three steps you run once, at startup; the last two repeat with every single question.

Step two usually causes the most trouble, so let us look at it in plain Python, with no library at all. Below you cut the text into pieces of one hundred characters each - but with an overlap of twenty characters, meaning every chunk starts twenty characters before the previous one ended.

1guide = "Lions are most active at dawn and just after dusk. " * 8
2
3size = 100
4overlap = 20
5chunks = []
6
7start = 0
8while start < len(guide):
9    chunks.append(guide[start:start + size])
10    start += size - overlap
11
12print(len(guide))
13print(len(chunks))
14print(chunks[0][-20:] == chunks[1][:20])

The result is

408

,

6

and

True

. That last line is the interesting one: the tail of the first chunk is literally the same text as the head of the second. That is what the overlap is for - with a clean cut, a sentence sliced in half would exist nowhere in one piece and the search would have no chance of finding it. Notice too that the loop step is

size - overlap

, that is eighty rather than a hundred: which is exactly why 408 characters produce six chunks and not five. In a real project you do not write this by hand - you use

RecursiveCharacterTextSplitter

, which additionally tries to cut on paragraph and sentence boundaries instead of in the middle of a word.

Back to LangChain, where we carry out steps one and two.

TextLoader

reads a text file from disk, and its

load

method returns a list of Document objects - not plain strings, but content together with metadata, such as the name of the source file.

RecursiveCharacterTextSplitter

takes the two parameters you have just met in their hand-made form:

chunk_size

is the length of a chunk in characters,

chunk_overlap

is the overlap.

1from langchain_community.document_loaders import TextLoader
2from langchain.text_splitter import RecursiveCharacterTextSplitter
3
4loader = TextLoader("safari_guide.txt")
5documents = loader.load()
6
7splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=200)
8chunks = splitter.split_documents(documents)

After those four lines

chunks

holds a list of a few dozen pieces of the field guide, each about a thousand characters long, with a two-hundred-character overlap - so twenty percent of the text repeats between neighbours. The method is called

split_documents

rather than

split_text

because it takes documents and returns documents, carrying the metadata over onto every chunk. None of this has gone to any model yet - it is a plain text operation, performed entirely on your own machine.

Step three: embeddings. An embedding is a list of numbers describing the meaning of a chunk - texts with similar meaning get vectors that sit close together, even when they share no words at all.

OpenAIEmbeddings

is the model that turns text into such a vector, and

FAISS

is an index that stores the vectors and can find the nearest neighbours of a given point in an instant.

1from langchain_openai import OpenAIEmbeddings
2from langchain_community.vectorstores import FAISS
3
4embeddings = OpenAIEmbeddings()
5vectorstore = FAISS.from_documents(chunks, embeddings)

The

from_documents

method computes an embedding for every chunk and puts it into the index. This is the only moment in the whole pipeline where you pay for processing the entire guide, and that is why you do it once, when the application starts, rather than on every tourist's question. Worth remembering: this is a completely different model from

llm

- the embedding model does not converse and does not generate sentences, it only measures meaning.

That leaves steps four and five, which LangChain ties into a single object.

as_retriever

turns the index into a search engine, and

search_kwargs

with its

k

parameter decides how many of the most similar chunks that engine returns.

RetrievalQA.from_chain_type

joins the search engine to the model, where

chain_type="stuff"

means the simplest strategy of all: stuff every retrieved chunk into the prompt at once.

1from langchain.chains import RetrievalQA
2
3qa_chain = RetrievalQA.from_chain_type(
4    llm=llm,
5    chain_type="stuff",
6    retriever=vectorstore.as_retriever(search_kwargs={"k": 3})
7)
8
9result = qa_chain.invoke({"query": "How do I watch lions safely?"})
10print(result["result"])

That single

invoke

does everything at once: the question becomes an embedding, the index picks the three closest chunks of the guide, they go into the prompt as context, and the model writes the answer. Let us be clear about what did not happen: the model was taught nothing, and a second later, asked without RAG, it will once again know nothing about your rulebook. All the knowledge arrived in the prompt, for one request only. You pass the input under the

"query"

key and collect the finished text from the

"result"

key - it is a dictionary, so

.content

will not work here.

The chatbot as a service

The console version works beautifully as long as you are the only user, sitting with a laptop in your tent. The moment the reserve's website and the guides' app both need Darwin, you need an HTTP interface.

FastAPI

is a web framework where routes are described with a decorator and the shape of the data with classes inheriting from

BaseModel

, out of the pydantic library. Such a class checks the types for you: if somebody sends

message

as a number, they get a readable error before your code even starts.

1from fastapi import FastAPI, HTTPException
2from pydantic import BaseModel
3
4app = FastAPI()
5
6class ChatRequest(BaseModel):
7    message: str
8    session_id: str
9
10class ChatResponse(BaseModel):
11    response: str
12
13@app.post("/chat", response_model=ChatResponse)
14async def chat(request: ChatRequest):
15    config = {"configurable": {"session_id": request.session_id}}
16
17    try:
18        answer = chatbot.invoke({"input": request.message}, config=config)
19        return ChatResponse(response=answer.content)
20    except Exception as e:
21        raise HTTPException(status_code=500, detail=str(e))

The important part is what did not change: the same

chatbot

object, the same

store

dictionary, the same

invoke

and the same

config

. The only thing that changed is where the question comes from - instead of

input

at the keyboard it arrives in the body of an HTTP request. The session identifier is now sent by the client, and it is the client that decides which pigeonhole of memory the conversation lands in; a browser may well keep that identifier in a cookie, but the minutes of the conversation still sit on your server and are still resent to the model with every request.

HTTPException

turns an exception into a proper 500 response instead of shipping a stack trace to the client. You start the server with

uvicorn main:app --reload

. Under heavier traffic it is also worth swapping

invoke

for

await chatbot.ainvoke(...)

, so that one slow model call does not block all the other requests.

Summary

• The chat API is stateless. A chatbot remembers context only because you store the message history and send it with every request - the model memorises nothing by itself, and browser cookies never reach it.
• The history is a list of dictionaries with the keys

  "role"

  and

  "content"

  , and there are three roles:

  "system"

  ,

  "user"

  and

  "assistant"

  .
• Appending a message is five pieces in this order:

  messages

  ,

  .append(

  ,

  {"role": "user", 

  ,

  "content": text}

  ,

  )

  .
• The LangChain memory import is

  from

  →

  langchain_community.chat_message_histories

  →

  import

  →

  ChatMessageHistory

  .

• RunnableWithMessageHistory

  wraps an existing chain and adds the history pointed at by the

  session_id

  from

  config

  all by itself.
• The order for building a chatbot: define the system prompt, configure memory, create the chain, implement the conversation loop.
• RAG is a technique for enriching a model's answers with information from documents. It is not a file format, not a type of neural network and not an API protocol.
• The RAG pipeline: load the documents, split them into chunks, create embeddings, search for similar chunks, generate the answer with context.

Take one image away from this lesson, @name: the model is a guide with a perfect eye and no memory whatsoever - you are the one keeping the expedition journal, and you are the one holding it open in front of him before every question, together with the right page of the reserve's field guide.