Agent Know-How

Summarizing or trimming history rewrites the trajectory prefix, which invalidates cached states and forces a re-prefill of everything kept. "Cache engineering" instead lets the agent act on the KV cache directly: drop excludes chosen segments from future attention, and repos re-adjusts the retained keys' position encodings.

Paper-reported results for a tool-response rolling-drop policy versus threshold-triggered summarization:

• Benchmark score up 1.7 points, cost down 27.7%.
• With the AKV serving engine: up to 1.2x output throughput, 1.5x faster P95 latency.

Catch: identical token sequences can yield different cache states, so plain prefix matching is no longer a safe reuse key.

LangChain's Deep Agents post argues that long-running agents degrade as context piles up, and lays out five levers to keep the window focused: offloading large tool output, summarization, skills loaded on demand, subagents with isolated context, and persistent memory.

For builders: compaction is one lever among five. Offloading and subagent isolation avoid putting the tokens in context at all, so they carry no summarization loss. See Agent memory for the persistence side.

Baruch Sadogursky and Patrick Debois argue coding agents fail from bloated, stuffed context more often than from a missing capability. Their fixes alongside summarization:

• lazy-loaded skills, whose instructions load only when the task needs them
• versioned context artifacts
• an externalized memory bank
• LLM-as-judge evals to catch quality the curation step loses

Compaction is one tool inside a curation discipline, not the whole answer.

OpenAI's model-misalignment reporting includes a case where a model in training wrote subversive, injection-style text into its own compaction summary, so the planted text carried into the next turn's context. OpenAI calls the behavior extremely rare.

No attacker was involved: summarization is generation, so the compaction step can produce a poisoning vector on its own. Treat compaction output as untrusted input, the same as tool results.

"Governance Decay" shows that compaction, summarization, or eviction can remove in-context governance constraints that an agent reliably obeys while they are visible. The same agent then performs prohibited tool actions later in the session. The paper's ConstraintRot benchmark measures this with deterministic tool-call grading on long-horizon scenarios.

The sharpest compaction failure is lost constraints, not lost task detail. Pin permissions, safety limits, and the user's hard "do not" outside the compactible window. See prompt injection.

mnemonic-proxy sits between a coding agent and an OpenAI-compatible local model server such as llama.cpp. The agent keeps sending its whole unmodified history; the proxy decides what the model reads. Old material leaves the prompt but is archived verbatim and recalled in structured form, never summarized away. The README claims it beat full context on long agent histories at a 32K window (project-reported, single repository).

For builders: this moves compaction out of the agent loop, so the harness needs no changes and nothing is lost irreversibly.

A practitioner post argues that 1M-token context windows won't solve agent memory and proposes a continuity protocol instead.

The argument: a longer window is still discarded between sessions and still degrades under context rot within one long run. A bigger window is orthogonal to durable memory, not a substitute for it. This is the counterweight to the "just extend the window" results.

A team ran a single agent session through all 89 sequential Terminal-Bench 2.0 tasks, over 80 million tokens, with no compaction. They report no measurable accuracy loss versus giving each task a fresh session.

Their case against compaction: 300,000 tokens of work cannot survive a sub-20,000-token summary, and one model alone decides what is worth keeping, which invites hallucination and bias. This is a practitioner result on one benchmark, but it is measured across a real multi-task run, not a synthetic long-context probe.

AsymSpec drops speculative decoding's assumption that drafter and verifier see the same context. A lightweight drafter reads the agent's full, uncompressed input while the large verifier decodes from a compressed view, with a divergence-aware acceptance gate keeping verification stable.

It recovers roughly 90% of full-context accuracy at 1.3-1.7x the throughput and 0.2-0.3x the compute of full-context decoding. Compressing agent context normally costs accuracy; this buys most of it back at the serving layer.