Agent memory definition
Agent memory is the external storage and retrieval layer that lets an AI agent carry information across turns and sessions, because the underlying language model keeps no state of its own between calls. It is usually split into short-term memory scoped to the current session and long-term memory that persists beyond it
Agent memory is how an AI agent remembers anything at all: the model itself forgets everything the moment a request finishes, so memory is a separate store that writes information down and reads it back into the next prompt. Memory should hold what actually happened with a user, not durable business facts. Durable facts about the business belong in governed content the agent retrieves, which is the split Sanity draws between agent-authored notes and retrieved content served through Context MCP.
AI agents need memory because large language models are stateless between calls. As IBM puts it, "Large language models (LLMs) cannot, by themselves, remember things. The memory component must be added." Every request is answered from whatever text you hand the model in that request, and once the response comes back, the model retains nothing.
So any continuity an agent appears to have is manufactured. Something outside the model recorded what happened, decided what was worth keeping, and put the relevant pieces back into the input on the next turn. That machinery is agent memory.
Not every agent needs it. IBM notes that a simple reflex agent, one that maps an input to an action with no reference to history, works without memory. Memory becomes necessary when the agent has to honor a preference stated three sessions ago, avoid repeating a failed approach, or pick up a task where it left off.
Agent memory is conventionally organized into five tiers. Amazon Web Services defines them in its Well-Architected Agentic AI Lens as follows, verbatim:
Short-term (session) memory is "transient context scoped to the current task or conversation, typically stored in fast in-memory stores with TTL-based expiration." Long-term (persistent) memory is "durable knowledge that persists across sessions, including learned preferences, historical outcomes, and domain knowledge." Episodic memory holds "records of specific past interactions that can be retrieved to inform current decisions." Semantic memory is "general domain knowledge stored as vector embeddings for semantic retrieval." Procedural memory holds "learned patterns for task execution that inform how agents approach recurring task types."
The episodic, semantic, and procedural split is not vendor branding, but a set of terms popularized in the CoALA paper, "Cognitive Architectures for Language Agents" by Sumers, Yao, Narasimhan, and Griffiths (arXiv:2309.02427, published in Transactions on Machine Learning Research in February 2024), which borrows the terms from cognitive science. That paper is why many vendors writing about agent memory use the same four words.
One clarification worth making: the "memory" feature consumer chat assistants expose to users, the one that remembers you prefer metric units, is cross-session semantic memory about preferences. It is one tier of five, not the whole concept.
No. The context window is the token budget a model can read in a single call, which is capacity, not storage. Nothing in the window survives the end of the call. Agent memory decides which durable information gets loaded into that window on a given turn, so a larger window does not create memory any more than a bigger desk creates a filing cabinet.
There is also evidence that filling a large window is not a substitute for choosing well. "Lost in the Middle: How Language Models Use Long Contexts" by Liu et al. (TACL, 2024) found a U-shaped performance curve: models do best when the relevant information sits near the beginning or end of the context, and they degrade when it is buried in the middle.
Cost points the same way. Mem0's 2025 paper (arXiv:2504.19413, submitted 28 April 2025) reports more than 90% token cost savings and 91% lower p95 latency against a full-context approach that replays the entire conversation history, measured on the LOCOMO benchmark. That is a vendor-authored result about the vendor's own product, so treat it as evidence for the general point rather than a ranking: replaying everything is expensive, which is why real memory systems extract, consolidate, and summarize instead.
Agent memory and retrieval-augmented generation (RAG) both fetch text and put it in a prompt, so they can look identical from the outside. The difference is authorship, scope, and lifecycle. RAG retrieves from a corpus that humans or upstream systems authored, which exists independently of any conversation: documentation, product catalogs, and policy documents. Agent memory is written by the agent or the session itself, is scoped to a user or a thread, and records what happened, not what is true.
The boundary is not a clean wall at the implementation level. IBM describes RAG as "one of the most effective techniques for implementing" long-term memory, and vector search over stored episodes is a common design. The useful distinction is what each store is for, not a claim that one cannot be built with the other's machinery.
The practical consequence is straightforward: a memory system that has to remember your return policy is storing a fact that will drift. Put facts about the business in a content system with review, versioning, and an owner, and let memory hold only what the agent learned about this particular user. Agent memory is also distinct from fine-tuning, which changes model weights and is slow, global, and hard to revoke. Memory is data outside the model, editable and deletable per record.
Agent memory is one of four kinds of context an agent needs each turn, and the other three have different owners and lifetimes. Agent memory is that fourth category. The guide's point is that "if your stack treats all four as one bucket, the wrong people end up making the wrong decisions."
Sanity is the Content Operating System for the AI era, and what it ships covers the two durable layers next to memory rather than memory itself. Retrieved content reaches agents through Context MCP, a hosted read-only endpoint any agent loop can connect to, using either GROQ mode, which queries the dataset at request time with no build step, or Knowledge Bases, a pre-built index over material you choose (currently in beta and opt-in). Static instructions can be modeled as a Sanity document, so the system prompt gets fields with separate owners, version history, and rollback. Nearform's team put it this way: "Storing the system prompt in a Sanity document is genuinely useful. Editors tuned the agent's voice without any code changes."
What it does is keep the facts out of memory, so memory only has to remember the conversation.
Explore Sanity Today
Understanding agent memory is just the beginning. Take the next step and discover how Sanity can enhance your content management and delivery.
Last updated: