Letta, an agent that manages its own memory

Letta, an agent that manages its own memory

Letta is a runtime for agents where memory is not a layer bolted on the side but something the agent edits itself by calling tools. It grew out of research on the MemGPT project and kept its core idea: treat the context window like working memory that has to be managed.

One thing needs stating immediately, because it changes the picture. The Python server, meaning the letta-ai/letta repository together with the letta package on PyPI, was described by its authors on 3 July 2026 as deprecated and kept in maintenance mode only. That package last shipped version 0.16.8 on 14 May 2026 and requires Python between 3.11 and 3.13. Development moved to Letta Code, the @letta-ai/letta-code package installed from npm, and to the Letta Agent SDK, while self-hosting now goes through the App Server. Both repositories carry the Apache 2.0 licence.

The idea described below survives the move, so it remains worth understanding. What changed is the thing you run: instead of a Python API server you get an agent harness in Node.js with the same memory blocks and the same background processing.

The key difference from memory libraries

Worth naming up front, since it decides whether reading further is worthwhile at all.

A memory library attaches to an agent you already have. You call its function before answering, receive facts, paste them into the prompt. Your code still drives the flow.

Letta inverts that. The agent is an entity on the server side, with its own identity, its own state, and its own decisions about when to write something to memory and when to use it. You converse with that agent rather than building a loop where the model is one of the steps.

The consequence is serious: you do not add Letta to your application, you build your agent inside Letta. That is a far larger commitment than adding a dependency, and precisely why comparing this with Mem0 or Zep as equivalent options misleads. They solve the same problem at entirely different levels.

Memory blocks

The basic element is a memory block: a named piece of text permanently sitting in the agent's context, which the agent can change.

Two are conventional. A block describing the person collects what the agent learned about them. A block describing the agent itself holds its role, style, and constraints. Beyond those you can add your own, a block with the current state of the case the agent is working on for instance.

The agent has tools for appending to and replacing block contents. That is the heart of the approach: memory changes as a result of the model's decision rather than a call from your code.

The advantage is real. An agent that heard the person changed jobs can correct the relevant block itself rather than waiting for an external layer to detect that change. No rules describing what is worth remembering are needed.

The drawback is equally real and needs knowing. A model managing memory does it imperfectly: it records irrelevant things, loses relevant ones, and sometimes overwrites something that was needed. Blocks have a size limit, so space runs out and then a decision follows about what falls away. Over longer work it pays to inspect block contents, since that is the only way to tell whether memory actually works.

Context paging

The second idea carried over from the research concerns what happens when a conversation does not fit the model's window.

The analogy is directly systemic. What sits in context resembles working memory: fast and limited. The rest lies in external storage the agent reaches with a tool call, exactly as a system reaches for a disk.

The agent therefore has tools for searching its own conversation history and archive. When it needs something not in context, it calls search and pulls a fragment back.

That mechanism solves a real problem while introducing a cost that is easy to miss. Every reach into the archive is an extra turn with the model, so an answer requiring two such calls takes three times as long as one without them. With an assistant holding a live conversation that is noticeable.

Know also that the model will not always reach for the archive when it should. If it decides it knows enough, it will answer from what it holds in context, and that answer will be confident and incomplete. That is the same class of problem as with any mechanism resting on a model's judgement.

Processing during idle time

The third idea moves memory tidying outside the moment of answering.

The point is that a separate agent works in the background: reviewing what happened, summarising, tidying blocks, and drawing conclusions. It happens when nobody waits for an answer.

The gain is twofold. The user's answer carries no memory work, so it arrives faster. And the tidying can be more thorough, since there is no time pressure and a stronger model can be assigned to it.

The price is token cost for work nobody ordered. A background agent consumes them regardless of whether anyone is talking to it, so across many agents that line grows in a way the conversation count does not predict. That is a thing to measure before running anything at scale.

Tools and what an agent can actually do

Memory is the loudest element here, while an agent managing only its own memory is of no use to anybody. The real work starts with tools reaching into your systems.

You define a tool as an ordinary Python function with a description, and the agent decides when to call it. The mechanism is the same as with memory, except the effects leave the agent: a database query, a message sent, a record changed.

Here comes something to think through earlier than in an ordinary application. The agent holds durable state and acts also when nobody is watching, so tool permissions must be constrained regardless of what the model read in a conversation. A tool deleting data or moving money requires confirmation on the system side rather than trust in the model's judgement.

The second thing concerns descriptions. The model sees only the name, the description, and the argument schema, so a sentence stating plainly when to call the tool beats one stating merely what it does. With a long lived agent that difference compounds, since a wrong tool choice enters the history and shapes later decisions.

The third is error handling. A tool that throws returns a message to the agent, and the agent decides what next from it. A message describing what went wrong and what could be done instead turns a failure into an intermediate step. A technical message usually leads to repeating the same attempt.

Many agents and dividing the work

Since an agent is a durable entity, the question of several at once arises naturally, and two arrangements deserve separating.

The first is one agent per user. Each person gets their own, with their own memory about themselves. That arrangement is obvious for assistants and equally the one growing linearly with user count, so a plan for deletion and archiving should exist before the first deployment.

The second is specialised agents sharing certain memory blocks. A shared block with company rules or a project's current state, visible to several agents, lets them work from the same knowledge without duplicating it. A change one makes is immediately visible to the rest.

That second arrangement is tempting and deserves caution. Shared state changed by several independent entities is a classic source of trouble, familiar from outside this field. If two agents can overwrite the same block at once, sooner or later they will.

The practical advice: keep shared blocks read only, and leave writing to one agent or to your own code. Sharing knowledge then becomes an advantage rather than a hazard.

Letta against the alternatives

The first row differs from the others in scope rather than quality. Choosing it means choosing how to build an agent, not merely how to hold memory. That suits a new project and causes trouble when adding memory to something already running.

The last row deserves honest consideration. For an assistant that must remember three things about a user, a database table with three columns beats every option above. These tools begin earning their keep where you do not know in advance what will be worth remembering.

Note too that background processing and context paging can be reproduced yourself. These are not closed mechanisms but ideas, and a simpler implementation is often enough for a particular project.

Running and deploying it

npm install -g @letta-ai/letta-code
letta server --backend local --listen ws://127.0.0.1:4500

That is the current route. You install the command line tool from npm, which needs Node.js 22.19 or newer, then start the App Server, a service your application talks to over WebSocket, most conveniently through the Letta Agent SDK. Running it yourself is fully available thanks to the Apache 2.0 licence, and the vendor cloud remains the alternative.

The older route, built on pip install letta and the Python API server, still works and still backs the first generation SDKs. Its repository sits in maintenance mode, though, so a new project has no reason to reach for it.

On the local backend the agent state, memory blocks included, lives on the machine running the server and is versioned with git. On the cloud backend the state stays with the vendor while the agent harness itself can run on your own hardware, which keeps tools and data close together.

Agent state is durable, so an agent created once persists and remembers earlier conversations. That sounds obvious and carries an operational consequence: agents accumulate, each takes space, and each holds its own history. In an application creating one agent per user, know from the start how to delete them and what happens to their memory on a data deletion request.

Inspecting what an agent holds in memory matters more here than in an ordinary application, since the model changes memory itself. Wiring in a tracing tool, Arize Phoenix for instance, lets you see which tool calls produced the current block state.

When this is the right choice

Three conditions deserve checking before deciding.

The first: you are building something new. With an existing application holding its own agent loop, moving to this runtime means a rewrite rather than adding a dependency.

The second: the agent is meant to live long and learn about the person. For one off tasks where every call is independent, the whole mechanism is a needless cost.

The third: you accept that a model manages the memory. That means consenting to some unpredictability in exchange for not having to describe rules about what to remember. Where you must know exactly what was recorded, for regulatory reasons for instance, that consent is sometimes impossible.

If any condition fails, a simpler memory layer or your own table delivers the same at lower cost and with more control.

Common mistakes

The first is treating this as a library to bolt on. It is an agent runtime, so the choice concerns the whole application's architecture.

The second is never inspecting memory blocks. A model manages them imperfectly, and without looking at the contents you will not learn it recorded trivia and lost what mattered.

The third is ignoring the cost of background processing. Tokens get consumed while nobody is talking, so the bill does not follow the conversation count.

The fourth is ignoring latency when reaching into the archive. Every such call is an extra turn with the model, which is felt in a live conversation.

The fifth is creating an agent per user with no plan for deleting them. Durable state means they accumulate and remain, including after a user leaves.

The sixth is assuming the agent will always reach for the archive when it should. If it decides it knows enough, it answers from context, confidently and incompletely.

FAQ

How does Letta differ from Mem0 and Zep?

In scope. Mem0 and Zep are memory layers you attach to your own agent and call from your code. Letta is a runtime where the agent lives on the server side and edits its own memory with tools, so the choice concerns architecture rather than a dependency.

What are memory blocks?

Named pieces of text permanently present in the agent's context, which the agent can append to and replace with its own tools. Conventional blocks describe the person and the agent itself, and custom ones can be added, for the current state of a case for instance.

Can Letta run on my own infrastructure?

Yes, the licence is Apache 2.0. The App Server handles this, started with letta server from the @letta-ai/letta-code package, and your application connects to it over WebSocket. The old Python API server from the letta package can still be self-hosted, but its repository has been in maintenance mode since July 2026.

Is this related to MemGPT?

Yes, the product grew out of research on that project and kept its core idea: treating the context window as working memory, with the rest held in external storage the agent reaches through a tool.

What does it cost?

Running it yourself is free; you pay for tokens at your model provider. A hosted variant with a free tier and paid plans also exists, and checking the current price list before deciding is worthwhile, since the background processing line grows independently of the conversation count.

Documentation sits on the product site, and the code and releases in the GitHub repository.