Memory in LangGraph, threads and a durable store

Memory in LangGraph, threads and a durable store

LangGraph splits agent memory into two layers that solve different problems and get confused: checkpoints tied to a thread, and a store operating across threads. The current release carries version 1.2.10, under the MIT licence, with required Python from 3.10.

Understanding that split is effectively the whole content of this text, since almost every memory problem in this library comes from using one layer where the other was needed.

Two layers and what each does

A checkpoint records the graph's state and belongs to a thread. A thread is one conversation or one task, identified by an identifier supplied on the call.

A durable store holds data meant to outlive a single conversation, identified by a namespace, usually tied to a user. The same person returning in a new thread still reaches what was stored there.

The symptoms of confusing them are characteristic and worth knowing.

If a user returns after a week, starts a new conversation, and the agent does not remember their preferences, you were storing them in graph state. State belongs to a thread, and the thread is new.

If the reverse, a conversation does not resume where it was interrupted, then either no checkpoint exists or you are supplying a different thread identifier from before.

A typical application needs both layers at once, and that is the right arrangement rather than a compromise. Checkpoints hold the current conversation's flow; the store holds what is known about the person.

Checkpoints in practice

from langgraph.checkpoint.postgres import PostgresSaver
from langgraph.graph import StateGraph

with PostgresSaver.from_conn_string(CONNECTION) as saver:
    saver.setup()
    graph = builder.compile(checkpointer=saver)

    result = graph.invoke(
        {"messages": [{"role": "user", "content": "let us continue"}]},
        config={"configurable": {"thread_id": "conversation-123"}},
    )

The thread identifier is the only thing deciding continuity here. Supplying the same one resumes from the last saved state; supplying a new one starts from nothing.

What matters is that the write happens after every graph step rather than at the end of a call. Three consequences follow, all worth knowing.

The first is fault tolerance. A process that died mid processing resumes from the last saved step after restart rather than starting over.

The second is the ability to pause a graph and wait for a human decision. The graph stops before a chosen step, the state is saved, and resumption can happen in an hour or in two days.

The third is size. Writing after every step means a conversation of twenty turns through a five node graph produces a hundred records rather than twenty. The database grows faster than the message count suggests, and a policy for deleting old threads is mandatory here rather than optional.

Choosing where to write

The available variants differ in purpose enough that choosing wrongly is costly.

The variant holding everything in process memory exists purely for development work. It works while the process lives and a second process cannot see it. Running that in production with two application instances produces a symptom that looks like conversations vanishing at random: the user lands on one instance, then the other.

The file database variant suits local experiments, since it survives a restart, and does not suit several processes at once.

Production calls for the variant backed by a relational database. It requires a one off schema preparation, and that is the step easily forgotten, since without it the first call fails with a missing table error.

Practical advice: set the production variant from the start, in the test environment too. The behavioural gap between process memory and a database is wide enough that bugs surface only after the switch, meaning at the worst possible moment.

The cross thread store

The second layer is simpler to use and harder to design.

Writing means supplying a namespace, a key, and a value. The namespace usually holds a user identifier, which keeps one person's data separate from another's.

store.put(("user", user_id, "preferences"), "language", {"value": "english"})
result = store.search(("user", user_id, "preferences"), query="communication language")

The difficulty sits elsewhere than in the interface. You have to decide what to record at all, and that is a product decision rather than a technical one.

Two approaches are sensible. The first is recording what the user stated outright: preferred language, contact hours, answer format. Simple, predictable, and sufficient in most cases.

The second is letting the model decide, by giving it a tool that writes to the store. Richer and less predictable, since a model records trivia and loses what matters. With that variant, inspecting store contents after a week of work is mandatory.

A third approach, recording everything said in a conversation, is tempting and almost always wrong. The store fills with noise, search returns meaningless fragments, and cost climbs.

What reaches the prompt

Memory itself is useless until it influences an answer, and that step gets neglected.

Checkpoints enter the model automatically, since message history is part of graph state. Here the problem is the reverse: on a long conversation the history stops fitting the window and has to be trimmed or summarised.

Simple trimming, keeping the last N messages, works in most cases and costs nothing. Summarising the older part gives more context and costs extra model calls. The choice depends on whether conversations genuinely reach back to things said fifty messages ago.

Store data does not enter the prompt by itself. You have to fetch it and insert it, and the same rule applies as with any memory: a few relevant facts help more than twenty arbitrary ones. Searching by the current question's content usually beats pulling a whole namespace.

Describe in the prompt where those facts came from, too. Sentences pasted without a heading get treated on a par with the user's question, while a section labelled as knowledge about the person yields clearly better results.

The thread identifier as a design decision

It looks like a technical detail and is one of the more consequential decisions when building on this library.

The choice reduces to what you consider one conversation. An identifier equal to the user identifier means every person holds one endless thread: the whole history always available, the whole history always growing. An identifier generated whenever a chat window opens means conversations stay separate and nothing carries between them.

Both variants have their place. A support assistant, where each report is a separate case, suits the second. A personal assistant, where continuity is the point of the product, suits the first, provided you have a plan for trimming history.

A third variant, most often the right one, is a session identifier with a limited lifetime: a new conversation after a few hours of inactivity, with whatever should survive going to the cross thread store. It combines predictable state size with continuity where it is needed.

Take that decision before the first deployment, since changing the identifier scheme afterwards makes existing threads unreachable. The history does not vanish from the database; nobody will ever land on it again.

Concurrency and what happens on two requests

One thread served by two parallel calls is a situation easy to overlook and more common than a design suggests.

The typical scenario: a user sends a message, does not see an answer, sends again. Or the interface retries a request after a timeout while the previous one is still running. Both calls work on the same thread and both write checkpoints.

The outcome depends on write order and can be hard to reproduce: some steps from one run, some from the other, and a final state matching neither. The symptoms look like random bugs in the agent's logic.

Two answers exist, both on your side. You can lock parallel calls on the same thread, rejecting the second with a readable message. Or you can generate a new identifier for a retried request, accepting that the flows separate.

With a streaming interface it also pays to ensure that a user dropping the connection does not leave the thread in an intermediate state. A graph halted midway records what it managed, and on resumption picks up from there, which is sometimes desirable and sometimes a surprise.

LangGraph against external solutions

Treat the first row as a starting point rather than a temporary measure. A store with namespaces and content search covers most real cases, and the data stays in your database alongside the rest of the application.

Reaching for external layers makes sense when you need something the built in one does not do: automatic fact extraction from conversation without writing your own logic, or resolving contradictions between facts from different moments. Those are specific needs rather than default ones.

The last row is a different kind of choice, since it concerns the whole application's architecture rather than a memory layer. With a working graph in this framework, moving there means a rewrite.

Time travel and resumption

Writing after every step provides a capability that is invaluable when diagnosing.

A thread's state history can be read and rewound to any point, then the graph run from there with different configuration or different input. Faced with "why did the agent make that decision", it lets you reconstruct the flow rather than guess.

The same capability supports pausing for a human decision. A graph stopped before a step performing an irreversible operation waits, and resumption follows approval, including from another process and after any delay.

Pair it with a tracing tool, LangSmith or Arize Phoenix for instance. Checkpoints show state while traces show exactly what went to the model and what came back. One without the other leaves a gap in reconstructing the flow.

Common mistakes

The first is the process memory variant in production. With two application instances the symptom looks like conversations vanishing at random.

The second is confusing the layers. User preferences stored in graph state disappear with the thread, and conversation history stored in the store does not resume the flow.

The third is having no policy for deleting old threads. Writing after every step makes the database grow far faster than the conversation count suggests.

The fourth is skipping the database schema preparation. The first call then fails with a missing table error, and the cause looks like a connection problem.

The fifth is sending the whole history to the model without trimming. On longer conversations that ends in exceeding the context window or a bill growing quadratically.

The sixth is writing everything a user said into the store. Search then returns noise, and with sensitive data the record outlives the conversation.

FAQ

How does a checkpointer differ from a store?

In scope. Checkpoints record graph state and belong to a thread, so they handle continuity within one conversation. A store holds data across threads, usually in a namespace tied to a user, so it handles what should survive between conversations.

Why does the agent not remember a returning user?

Most often because the information was written into graph state rather than the store. State belongs to a thread, and a returning user usually starts a new thread, so they receive an empty history.

Which write variant should production use?

The one backed by a relational database, since it survives a restart and works across several processes. The process memory variant exists for development work, and the file one for local experiments on a single process.

How do I handle a long conversation history?

By trimming to the most recent messages or summarising the older part. Trimming is cheap and sufficient in most cases; summarising gives more context at the price of extra model calls.

Do I need an external memory layer?

Usually not to begin with. The built in store with namespaces covers typical cases, and the data stays in your database. Mem0 or Zep are worth reaching for when you need automatic fact extraction or resolution of contradictions over time.

The persistence documentation sits on the project site, and the code and releases in the GitHub repository.