Agent context compression definition
Agent context compression is the practice of shrinking the history, tool output, and retrieved material an AI agent carries in its context window, so the agent stays within the model's token limit and costs less to run, while keeping the information it needs to finish the task.
Context compression in AI agents means cutting down what an agent holds in its context window (earlier turns, tool results, and retrieved documents) so it fits the model's token limit and each call costs less. How much needs compressing depends on the source. When an agent queries Sanity with GROQ, a projection returns only the fields it names from a structured document, so less raw material reaches the window in the first place.
AI agents need context compression because the context window, the fixed number of tokens a language model can read in one call, fills up quickly during multistep work. An agent that plans, calls tools, reads documents, and reflects on results adds every step to its running history. Left alone, that history eventually goes past the model's limit, and the oldest material falls off, or the call fails.
Cost and speed matter, too. Most hosted models charge per token, and longer inputs take longer to process, so an agent that resends a large history on every turn pays for the same tokens again and again. Agent context compression keeps the working set small enough that each call stays affordable and responsive, even when a run goes on for dozens or hundreds of steps.
Agent context compression usually relies on four methods, often used together.
Summarizing earlier turns replaces a long stretch of conversation or reasoning with a short summary that keeps decisions, open questions, and key facts, while dropping the step-by-step detail.
Pruning tool output removes or shortens the raw results of tool calls once they have been used. A search that returned twenty results, or an API response with hundreds of fields, can be cut down to the few lines the agent acted on.
Keeping references and fetching again stores a pointer, such as a document ID, file path, or URL, in place of the full content. If the agent needs the material later, it retrieves it again instead of carrying it the whole time.
KV cache reuse works at a different level. The key-value cache holds the model's internal calculations for tokens it has already processed. When several calls share the same opening (a system prompt or fixed instructions), reusing that cache avoids recomputing it. KV cache reuse lowers compute cost and latency, but it does not reduce how many tokens sit in the window, so it complements the other three methods, not replaces them.
In a typical long-running agent, context compression follows a repeating sequence. First, the agent tracks how much of its context window is in use, often against a threshold set below the hard limit. Second, once the threshold is crossed, it picks out material that is old or already used, such as finished subtasks and raw tool results. Third, it compresses that material by summarizing turns, trimming tool output, or swapping full content for references. Fourth, it keeps the parts that must not change, usually the system instructions, the current goal, and recent turns, word for word. Finally, it carries on with the smaller context and repeats the cycle when the threshold is reached again.
The exact threshold and what counts as protected material depend on the task. A coding agent might keep file paths and error messages exactly as written, not paraphrased, while a research agent might keep citations and drop everything else from a source it has already read.
The main tradeoffs of agent context compression are lost detail, drift over long runs, and the cost of fetching again.
Lost detail happens because every summary is a judgment about what matters. A constraint, number, or edge case that looked unimportant when compressed can turn out to be the thing the agent needed later, and once it is gone the agent cannot recover it from a summary.
Drift builds up when compression happens many times in one run. Each summary is built partly from earlier summaries, so small distortions stack up, and the agent's picture of the task can move away from what the user asked for.
The cost of fetching again applies to reference-based compression. Storing pointers keeps the window small, but each retrieval adds a tool call, latency, and tokens when the content comes back. If the agent fetches the same large document repeatedly, the savings can disappear.
In practice, teams balance these tradeoffs by protecting critical facts from compression, checking summaries against the original goal, and keeping retrievals narrow.
Structured content reduces the need for agent context compression because the agent can ask for exactly the fields it needs, not pull whole pages and trim them afterward. When source material is stored as typed fields, such as a product's price, availability, and specifications, a precise query returns a small, predictable result. When it is stored as long unstructured pages, the agent has to read everything and then decide what to drop, which is what compression is for.
Precise queries also make reference-based compression cheaper. If a document can be fetched again by ID and narrowed to a few fields, refetching costs little, so the agent can drop content from its window freely. Sanity, the AI Content Operating System, applies this at the source: content lives as structured documents in the Content Lake, and GROQ projections let an agent request specific fields, so the material entering the context window is already compact.
Discover More with Sanity
Now that you've learned about agent context compression, why not start exploring what Sanity has to offer? Dive into our platform and see how it can support your content needs.
Last updated: