Agent Builder Foundations
Mechanisms, math intuition, evidence, and application
Agent Builder Foundations
Mechanisms behind reliable agents
Durable explanations for builders who want to understand why a prompt, retrieval setup, tool call, memory system, or eval behaves the way it does. Each page separates paper-backed mechanisms, benchmark results, field reports, and LLM Digest synthesis.
13 concepts·7 clusters·evidence-tiered
Prompting and instruction following
What makes a prompt reliable?
Reliable prompts reduce ambiguity, constrain outputs, and make failures measurable.
Retrieval and grounding
Why does RAG accuracy degrade as the knowledge base grows, and what fixes it?
Naive top-k vector retrieval treats every chunk as an independent nearest-neighbor hit, so as a knowledge base grows, questions that need two or more chunks combined become steadily less likely to get everything they…
Tool use and agents
Why do reusable skills sometimes make an agent worse?
Grading a procedural skill by average task-success improvement hides its cost: the best-performing skills win mainly by regressing less on tasks the agent already solved, not by solving more — and most regressions…
Why did MCP go stateless, and what does that change for scaling agent tool gateways?
MCP's 2026-07-28 spec dropped the session-handshake header that pinned a client to one server instance, so any gateway node can now handle any request — the same statelessness trade that let HTTP scale horizontally,…
Why does forcing structured output make my agent stop calling tools?
Turning on JSON Schema constraints and tool calling at the same time can silently suppress tool calls in open-weight models — the schema is compiled into a token mask that makes tool-call tokens unreachable during…
Memory and context
Why does adding more context sometimes hurt an agent?
Most production agent failures trace back to unmanaged context, not weak reasoning — treating context as a lifecycle to architect, ingest, scope, anticipate, and compact (not a log to truncate when it gets too big) is…
Can you trust what your agent remembers?
Persistent agent memory is a write-once, replay-many attack surface — 2026 benchmarks show attackers can forge an agent's own reasoning history or plant poisoned facts through routine content like email, both with high…
Evals and reliability
Does a high benchmark score predict production reliability?
A benchmark pass rate measures one round of scoring against a fixed task set — 2026 evidence shows agent-optimization gains that look real on that single round can fail to transfer or even regress once the agent is…
Can you trust an LLM-as-judge score?
An LLM judge is a measurement instrument with its own biases, not ground truth — validate it the same way you validate the agent it grades, and for agent trajectories with checkable evidence, consider a deterministic…
Cost, latency, and operations
How does speculative decoding speed up LLM inference?
Speculative decoding drafts several tokens cheaply, then verifies them in one parallel pass through the target model — cutting decode latency without changing the output distribution, as long as the draft's guesses are…
Safety and control
Why can a tool designed to block exfiltration still leak your data?
Blocking the obvious exfiltration path — a model encoding secrets straight into a URL it fetches — isn't enough. Claude's web_fetch tool blocked exactly that, but a researcher still exfiltrated a user's name, city, and…
Does compacting an agent's context put its safety rules at risk?
Context compaction is not just a lossy cost optimization — a 1,323-episode benchmark shows it can silently erase the governance constraints a long-running agent was given, and only pinning those constraints outside the…
What actually breaks when you run MCP in production, and how do you defend it?
An analysis of documented MCP CVEs found most incidents cluster in four layers — unsafe tool execution, unauthenticated management endpoints, unrestricted outbound calls, and undetected tool-definition drift — and a…