Agent memory that does not wreck your prompt cache

Most advice on agent memory treats it as a storage question. Pick a vector store, summarize when the window fills, move on. That misses the bill.

Memory and prompt caching pull against each other. Every time you rewrite the front of your prompt, you throw away a cached prefix and pay to build a new one. On Claude Opus 5 that swap costs 12.5 times more per token than the cache read it replaced.

Why memory is a caching problem, not a storage problem

Prompt caching is a discount for resending an identical prefix. The provider keeps the processed prefix warm and charges a fraction to read it back.

The catch is in the word prefix. Anthropic's prompt caching reference states the rule directly: place the cache breakpoint on the last block whose prefix is identical across the requests you want to share a cache. Change any block at or before that point and the next request hashes differently.

Agent memory lives exactly there. Your user facts, running summary and task state all want to sit high in the prompt, near the system instructions. That is the most expensive place in the request to edit.

What a cache miss actually costs

Here are the published multipliers, with Claude Opus 5 prices worked through at a $5 per million token base rate.

Operation	Multiplier	Opus 5 price per million tokens
Cache read or refresh	0.1x	$0.50.
Five-minute cache write	1.25x	$6.25.
One-hour cache write	2x	$10.00.
Uncached input	1x	$5.00.

Read that table as a ratio rather than a price list. Going from a read to a five-minute write is 12.5 times the cost for the same tokens. Going to a one-hour write is 20 times.

Now picture a support agent with a 40,000 token prefix. Served from cache, that prefix costs about two cents per turn. Rewritten every turn because you keep updating a memory block near the top, it costs about 25 cents per turn on the five-minute tier. The cheapest AI API is not the cheapest to run, and this is where the gap opens.

Your memory block may be too small to cache at all

There is a floor, and it varies more than most people expect. The documented minimum cacheable prefix is 512 tokens on Claude Opus 5, 1,024 tokens on Claude Sonnet 5 and Claude Opus 4.8, and 4,096 tokens on Claude Haiku 4.5.

Nothing errors when you fall short. The request succeeds and simply is not cached, so you find out from your invoice.

That matters for a common pattern. Teams route cheap background summarization to a small model, then wonder why caching never helps there. On Haiku 4.5 a short system prompt plus a compact memory block can sit under 4,096 tokens and never qualify.

You also get at most four cache breakpoints per request. Treat them as a budget, not a decoration.

Clearing beats summarizing more often than you think

Two different tools get called "memory management", and they behave very differently.

Context editing is a way to remove specific content while leaving the conversation structure intact. Compaction is a pass that replaces the history with a generated summary.

The context editing documentation describes a tool-result strategy that clears the oldest tool results in order and swaps in placeholder text. Its default trigger is 100,000 input tokens, and it keeps the three most recent tool uses. You can exclude named tools, set a minimum amount to clear, and choose whether tool inputs go too.

Compare that to a summary pass. Clearing removes bulk you can identify, keeps the shape of the conversation, and costs no extra model call. Summarizing spends a model call, rewrites history into prose, and permanently loses whatever the summarizer judged unimportant.

For a tool-heavy agent, such as one that reads a dozen files before answering, the bulk is nearly always old tool output. Clear that first. Reach for a summary only when clearing is not enough.

The state injection trick worth knowing

Here is a specific fix for the most common cache killer.

You often need to change an instruction mid-run. A mode switch, a fresh policy, or a piece of state the agent must respect from now on. The obvious move is to edit the top-level system prompt, and that is the single most destructive thing you can do to a cache.

The documented alternative is to append a system message to the message array instead of editing the system field. The cached prefix stays byte-identical, so the discount survives, and the instruction still carries operator authority.

Two caveats. This is model-gated, so check support before you rely on it. And the message must follow a user turn rather than opening the conversation.

What survives outside the window

Not everything you call memory is durable, and some of it expires without telling you.

Microsoft documents that vector stores created through a thread, such as those made from message attachments, expire seven days after they were last active. That default exists to cap storage cost. If your idea of long-term memory is an attachment store from three weeks ago, Azure's file search reference says it is already gone.

Letta's write-up on agent memory frames the durable version well. It splits memory into a message buffer, in-context core memory blocks the agent edits itself, and external recall and archival stores reached by tool call. The blocks carry a label, a description and a character limit, which is what keeps them from growing into the whole context window.

Frameworks are converging on the same shape. Agent Framework's feature overview lists context compaction, memory and file access as built-in parts of its harness agent rather than things you assemble.

MCP went stateless. Your server still has state. Memory is the same story again: the transport got simpler, and deciding what persists became your design problem.

A layout that keeps the discount

Order the request so the volatile parts sit at the back.

1. Tool definitions first, in a fixed order. A reordered tool list invalidates everything behind it.
2. Frozen system instructions next. No timestamps, no request IDs, no per-user text.
3. Stable long-term memory after that, and only rewrite it on a real change rather than every turn.
4. Cache breakpoint here. This is the last block you expect to be identical next turn.
5. Volatile state and the current question last, including any mid-conversation system message.

Then verify rather than assume. Check the cache read token count on responses; if it stays at zero across turns that should share a prefix, something above your breakpoint is changing.

The decision, in one line

If your agent runs long conversations, clear old tool results before you summarize, keep everything volatile behind your last cache breakpoint, and inject mid-run instructions as a message rather than a system prompt edit.