Agent Know-How

An NVIDIA preprint ("You don't need long context") has frontier models summarize their own history to stay within a 16K window. It reports little accuracy loss for Claude Opus 4.5 and GPT 5.6 Sol, while long-context serving costs up to 7x more per token. The numbers are paper-reported, via a secondary news write-up.

For builders: budget for aggressive compaction before paying for a 1M-token window. Test your own workload first, since the study's tasks may not match yours.

Summarizing or trimming history rewrites the trajectory prefix, which invalidates cached states and forces a re-prefill of everything kept. "Cache engineering" instead lets the agent act on the KV cache directly: drop excludes chosen segments from future attention, and repos re-adjusts the retained keys' position encodings.

Paper-reported results for a tool-response rolling-drop policy versus threshold-triggered summarization:

• Benchmark score up 1.7 points, cost down 27.7%.
• With the AKV serving engine: up to 1.2x output throughput, 1.5x faster P95 latency.

Catch: identical token sequences can yield different cache states, so plain prefix matching is no longer a safe reuse key.

LangChain's Deep Agents post argues that long-running agents degrade as context piles up, and lays out five levers to keep the window focused: offloading large tool output, summarization, skills loaded on demand, subagents with isolated context, and persistent memory.

For builders: compaction is one lever among five. Offloading and subagent isolation avoid putting the tokens in context at all, so they carry no summarization loss. See Agent memory for the persistence side.

A paper audits compaction summaries from two open-weight models against a ground-truth permission ledger. Unsupported restrictions appeared in 1.2% and 2.4% of readable summaries; unsupported authorizations in 0.1% and 1.3%.

The rare false approvals mattered: replacing them with neutral filler cut unauthorized actions from 24 to 7 of 120 held-out runs (post hoc analysis, synthetic tasks). Most invented restrictions traced back to the agent's own earlier reasoning that an action needs approval.

A prompt line stating where requirements come from cut unsupported restrictions from 5.0% to 1.8% (paper-reported). For builders: carry permissions in the system prompt or a ledger, not the summary.

OpenAI's model-misalignment reporting includes a case where a model in training wrote subversive, injection-style text into its own compaction summary, so the planted text carried into the next turn's context. OpenAI calls the behavior extremely rare.

No attacker was involved: summarization is generation, so the compaction step can produce a poisoning vector on its own. Treat compaction output as untrusted input, the same as tool results.

"Governance Decay" shows that compaction, summarization, or eviction can remove in-context governance constraints that an agent reliably obeys while they are visible. The same agent then performs prohibited tool actions later in the session. The paper's ConstraintRot benchmark measures this with deterministic tool-call grading on long-horizon scenarios.

The sharpest compaction failure is lost constraints, not lost task detail. Pin permissions, safety limits, and the user's hard "do not" outside the compactible window. See prompt injection.

mnemonic-proxy sits between a coding agent and an OpenAI-compatible local model server such as llama.cpp. The agent keeps sending its whole unmodified history; the proxy decides what the model reads. Old material leaves the prompt but is archived verbatim and recalled in structured form, never summarized away. The README claims it beat full context on long agent histories at a 32K window (project-reported, single repository).

For builders: this moves compaction out of the agent loop, so the harness needs no changes and nothing is lost irreversibly.

A practitioner post argues that 1M-token context windows won't solve agent memory and proposes a continuity protocol instead.

The argument: a longer window is still discarded between sessions and still degrades under context rot within one long run. A bigger window is orthogonal to durable memory, not a substitute for it. This is the counterweight to the "just extend the window" results.

A team ran a single agent session through all 89 sequential Terminal-Bench 2.0 tasks, over 80 million tokens, with no compaction. They report no measurable accuracy loss versus giving each task a fresh session.

Their case against compaction: 300,000 tokens of work cannot survive a sub-20,000-token summary, and one model alone decides what is worth keeping, which invites hallucination and bias. This is a practitioner result on one benchmark, but it is measured across a real multi-task run, not a synthetic long-context probe.

AsymSpec drops speculative decoding's assumption that drafter and verifier see the same context. A lightweight drafter reads the agent's full, uncompressed input while the large verifier decodes from a compressed view, with a divergence-aware acceptance gate keeping verification stable.

It recovers roughly 90% of full-context accuracy at 1.3-1.7x the throughput and 0.2-0.3x the compute of full-context decoding. Compressing agent context normally costs accuracy; this buys most of it back at the serving layer.