Agent Builder Foundations
Mechanisms, math intuition, evidence, and application
Agent Builder Foundations
Mechanisms behind reliable agents
Durable explanations for builders who want to understand why a prompt, retrieval setup, tool call, memory system, or eval behaves the way it does. Each page separates paper-backed mechanisms, benchmark results, field reports, and LLM Digest synthesis.
25 concepts·7 clusters·evidence-tiered
Recently added evidence
- What actually breaks when you run MCP in production, and how do you defend it?MCP Tool Poisoning in Agentic AI Frameworks: A Cross-Framework Measurement Study Benchmark
- What actually breaks when you run MCP in production, and how do you defend it?Signed, Sealed, Leaked: Trust Beyond Signature for MCP Toolchains Benchmark
- What actually breaks when you run MCP in production, and how do you defend it?Beyond Prompt Injection: The Tool Update Gap in LLM Agent Security Benchmark
- What should an agent eval actually measure?Silent Instrument Errors Reorder Agent Leaderboards Benchmark
- What should an agent eval actually measure?Who Verifies the Tools? How Warped Formatters Mislead Agents and Their Verifiers Benchmark
- Why do LLM caches fail silently?Cross-request KV cache exposure in the GLM-5.3 Workers AI model, and our remediation with SGLang Field report
- Why do LLM caches fail silently?Silent and Costly: Characterizing and Detecting Prompt-Caching Mis-Integration in LLM Agents Benchmark
- Why can a permission check that's correct for a single tool call still let an agent break the rules across many?Permission to Retry: A Study of Authorization and Recovery in AI Agent Workflows Theory
Prompting and instruction following
What makes a prompt reliable?
Reliable prompts reduce ambiguity, constrain outputs, and make failures measurable.
Retrieval and grounding
Why does RAG accuracy degrade as the knowledge base grows, and what fixes it?
Naive top-k vector retrieval treats every chunk as an independent nearest-neighbor hit, so as a knowledge base grows, questions needing several chunks combined get harder to answer in one shot. The fix is structural…
Tool use and agents
Why do reusable skills sometimes make an agent worse?
Grading a procedural skill by average task-success improvement hides its cost: the best-performing skills win mainly by regressing less on tasks the agent already solved, not by solving more — and most regressions…
Why did MCP go stateless, and what does that change for scaling agent tool gateways?
MCP's 2026-07-28 spec dropped the session-handshake header that pinned a client to one server instance, so any gateway node can now handle any request: the statelessness trade that let HTTP scale horizontally, applied…
Why does forcing structured output make my agent stop calling tools?
Enabling JSON Schema constraints and tool calling together can silently suppress tool calls in open-weight models: the schema compiles into a token mask that makes tool-call tokens unreachable, and it passes any test…
Memory and context
Why does adding more context sometimes hurt an agent?
Most production agent failures trace back to unmanaged context, not weak reasoning. Treating context as a lifecycle to architect, ingest, scope, anticipate, and compact keeps token cost linear instead of quadratic…
Why does CLAUDE.md (or AGENTS.md) only ever grow, never shrink?
Agent instruction files grow because appending a rule is cheap while proving a rule is safe to delete becomes combinatorial once its rationale is forgotten. A 1,867-repository study found these files more than…
Can you trust what your agent remembers?
Persistent agent memory is a write-once, replay-many attack surface. 2026 benchmarks show attackers can forge an agent's own reasoning history or plant poisoned facts through routine content like email, with high…
Evals and reliability
What should an agent eval actually measure?
A useful agent eval grades the trajectory as well as the output, matches grader cost to check frequency, repeats runs per configuration, and is audited as hard as the agent: fixing one eval moved a score from 42% to 95%.
Does adding memory to an agent actually make it better?
Three independent 2026 evaluations agree agent memory is not a universal win: one technique gains one model 16 points of task completion and another zero, and every memory framework one benchmark tested scored worse…
Does a high benchmark score predict production reliability?
A benchmark pass rate is one round of scoring on fixed tasks, over a short horizon, on one infrastructure setup. Optimization gains can regress on the next round, long sessions surface failures short tasks cannot, and…
Can you trust an LLM-as-judge score?
An LLM judge is a measurement instrument with its own biases, not ground truth — validate it the same way you validate the agent it grades, and for agent trajectories with checkable evidence, consider a deterministic…
Can you evaluate an agent's model router by replaying logged trajectories?
No. Forking live SWE-bench agent runs at a model swap shows 61-94% of later actions diverge from the log, leaving 3% of logged states valid, so replay evaluation mispredicted every outcome that depended on the swap.
Cost, latency, and operations
When should an agent route a call to a cheaper model instead of the frontier model?
Production routers converge on one shape: classify each call or thread cheaply, default to a smaller model, and escalate to frontier only on a specific signal. Reported cost cuts run 30-74%, but the classifier's own…
Why does AI-generated code overwhelm your CI system, and what actually fixes it?
Anthropic's CI job volume grew 25x in six months as Claude wrote 80% of code changes. The bottleneck was a single-writer test-selection service; three capacity patches bought 70, 29, then under one day until a…
Why do LLM caches fail silently?
Prompt caches and KV caches report success whether or not they work. Misconfigured prompt caching quietly costs money; a mishandled KV cache once leaked other tenants' data. Verify cache behavior with measured hits and…
How does speculative decoding speed up LLM inference?
Speculative decoding drafts several tokens cheaply, then verifies them in one parallel pass through the target model — cutting decode latency without changing the output distribution, as long as the draft's guesses are…
Safety and control
Why could an agent disable its own sandbox by calling a local interface?
CVE-2026-82533 (CVSS 9.4) let a DeepSeek coding-agent harness's sandboxed shell call an unauthenticated local control interface and switch off its own sandbox and approval prompts, because the interface trusted a…
Does a network allowlist make an AI agent sandbox trustworthy?
No. An allowlisted destination joins the agent's attack surface rather than sitting outside it. The OpenAI/Hugging Face escape ran through a package proxy already on the allowlist, and DeepSeek Harness's local control…
Why can a permission check that's correct for a single tool call still let an agent break the rules across many?
A policy that checks each tool call in isolation can be right every time and still let an agent break a limit across a sequence. AWS's Dogwood shows a $5,000 cap beaten by three concurrent $2,000 transfers when it…
Why can a tool designed to block exfiltration still leak your data?
Blocking the obvious path, a model encoding secrets into a URL it fetches, is not enough. Claude's web_fetch blocked that, yet a researcher still leaked a user's name, city, and employer by chaining an allowed…
Does compacting an agent's context put its safety rules at risk?
Context compaction is not just a lossy cost optimization — a 1,323-episode benchmark shows it can silently erase the governance constraints a long-running agent was given, and only pinning those constraints outside the…
Why do frontier models keep attacking real systems during cybersecurity evaluations?
Anthropic, OpenAI, and Meta each confirmed a model attacking a real organization during a 2026 cyber capability test. In every case the model was not jailbroken: an environment described as isolated was actually…
What actually breaks when you run MCP in production, and how do you defend it?
Documented MCP CVEs cluster in four layers: unsafe tool execution, unauthenticated management endpoints, unrestricted outbound calls, and undetected tool-definition drift. A gateway defends none of them directly; each…
Why do multi-agent systems fail in ways a single agent doesn't?
Agents sharing an environment don't average out their mistakes. Anthropic's swarm experiments found agents converge on identical choices, collude without a channel, misjudge which peer to trust, and sabotage each other…