Your AI agent gets worse long before the context fills

Context rot is measurable, not folklore

Context rot is the observed decline in a model's ability to use information as the number of tokens in its window grows. It starts well before the window is full, which is why "we still have headroom" is a bad diagnosis.

Researchers built a benchmark specifically for this. LOCA-bench, published February 8, 2026, extends agent context toward infinity in a controlled way while holding the task fixed. Its finding is direct: agent performance generally degrades as environment state grows, and context management techniques substantially improve success rates.

The second half of that sentence matters as much as the first. Degradation is not a law of nature you accept. It responds to engineering.

Budget tokens per step, not per session

Set a target for what the model sees on any single step, and enforce it. Here is a workable starting budget for a coding or research agent, sized at 40,000 tokens per step:

Slot	Token budget per step	Notes
System instructions and rules	2,000-4,000	Stable, cacheable, rarely edited
Tool definitions	4,000-6,000	Trim unused tools; every schema costs tokens
Task state and plan	4,000	The current goal, not the history of goals
Retrieved material	12,000-16,000	Loaded for this step only
Recent turns and tool output	12,000	Older output gets cleared or summarised

Sourcegraph's practical guide to context engineering makes the same structural point: an agent at step 47 is deciding with the residue of steps 1 through 46 still in view. A budget is what stops that residue growing without limit.

The exact split matters less than having one. Without a budget, tool results accumulate silently, and the agent that worked in testing degrades in week three of production.

Three ways to reclaim tokens, and what each costs

Every context strategy trades one resource for another. A 2026 review of agent externalisation covers the same ground across memory, skills, protocols and harness design (arXiv 2604.08224). Pick with the cost in mind.

1. Just-in-time loading. The agent keeps lightweight references, such as file paths or record IDs, and loads content only when a step needs it. Anthropic recommends this as the default for long-running agents. The cost is latency, because each load is a round trip.
2. Structured note-taking. The agent writes notes to external storage and reads them back on demand. This preserves progress across long tasks. The cost is a retrieval hop, plus the risk that notes drift from reality.
3. Summarisation. Older turns are compressed into a shorter representation. The cost is information loss, and it is not evenly distributed: decisions and constraints survive, while exact identifiers and edge-case details tend not to.

These three appear in Anthropic's guidance on long-running agents as the standard toolkit. A fourth technique is worth naming because it costs almost nothing: deleting stale tool output. Clearing a 20,000-token file dump that the agent already extracted three lines from is pure gain.

Tool definitions are the budget line nobody audits

Tool schemas sit in every request, on every step, for the whole session. An agent with 30 tools loaded pays for 30 tool definitions at step 1 and at step 90.

Two fixes work well. Load tools per phase, so a research phase carries search tools and an editing phase carries file tools. And keep descriptions tight, since a verbose schema is charged on every call.

There is a security reason to trim as well. A February 2026 comparative threat model of agent protocols identified 12 protocol-level risks across MCP, A2A, Agora and ANP, including tools invoked from the wrong provider in multi-server setups. Fewer loaded tools is a smaller surface in both senses.

Why a bigger window is the wrong purchase

Buying a longer context window treats a quality problem as a capacity problem. The window sets an upper bound. It does not set the quality curve inside that bound. Practitioner write-ups reach the same conclusion, including one arguing that more tokens make agents worse.

For example, an agent that fails at step 45 with 180,000 tokens in context will usually fail at step 45 with 900,000 tokens available, because the noise ratio at that step is unchanged. The step budget is what moved, and it moved in the wrong direction.

Cost points the same way. Long contexts are billed on every step of the loop, so a bloated prefix multiplies across the run rather than adding to it once. The cheapest AI API is not the cheapest to run for exactly this reason: per-token price is not per-task cost.

Prompt caching interacts here too. Agent memory that does not wreck your prompt cache is the companion problem: this article decides what belongs in the prefix, and caching decides how much you pay to keep it stable.

What to change this week

Three changes, in order of payoff:

1. Log tokens per step, split by slot: instructions, tools, retrieval, history. You cannot budget what you do not measure. Developer discussion of agent harnesses has shifted to exactly this, judging by the Hacker News thread on effective context engineering.
2. Clear tool outputs the agent has already used, rather than carrying them to the end of the run.
3. Move your largest retrieval source behind a just-in-time load, and measure both quality and latency for a week.

If quality improves and latency rises, that is the trade working as designed. If neither moves, your bottleneck is elsewhere, such as a loop that never terminates. Your agent loops forever. It is probably tool_choice — worth ruling out before you rebuild your retrieval layer.