Agent Builder Foundations

An agent eval is an input plus grading logic, and the grading logic is usually the weak link. Three grader types trade cost against accuracy: code-based graders are cheap and objective but only check what you encoded; model-based graders handle nuance but are non-deterministic and expensive; human graders set the bar but don't scale.

Because agents vary run to run, one pass/fail number isn't enough. pass@k asks whether the agent can solve the task at all; pass^k asks whether it solves it every time. And a bad score on either can mean the eval, not the agent, is broken.

Treat an eval as software with the usual failure surface: bugs, ambiguous specs, and flaky behavior. All three produce the same symptom as a real agent failure — a low score.

• Grader choice is a cost/accuracy trade-off. Code-based graders (unit tests, string checks, static analysis) can run on every commit but only check what you encoded. Model-based graders catch nuance but add their own non-determinism; see can you trust an LLM-as-judge score?. Human graders calibrate the others. Real suites mix all three, matched to how often each check must run.
• One score hides whether failure is rare or reliable. An agent that solves a task 1 of 5 times and one that solves it 4 of 5 times both fail pass^5. The second is near production-ready behind a retry; the first is not solving the task.
• The unit under test is the system. Task information, tools, budget, and model all move the score, so a result describes the configuration, not the model alone.

Start from real failures. Anthropic recommends 20-50 tasks drawn from cases that actually went wrong. Each needs an unambiguous spec and a reference solution; ambiguity produced the 42%-to-95% CORE-Bench jump. Include negative cases where the right behavior is to refuse or say it doesn't know; solvable-only suites cannot detect overconfidence.

Keep the environment stable and grade outcomes. A flaky sandbox produces noise indistinguishable from a regression. Exact step matching penalizes a correct result reached by a different path, so grade the outcome and, separately, the trajectory.

Grade the artifact each agent type produces.

• Coding: unit tests for the outcome, a transcript pass for process.
• Conversational: backend state verification plus an LLM rubric for tone.
• Research: groundedness, coverage, and source quality as separate checks.
• Computer use: interface state (DOM, screenshots) and backend state, since a correct-looking screen can hide a failed action.

Measure the configuration, repeatedly. In an 18,000-trajectory study of coding agents, 54% of outcome variance was run-to-run noise within identical configurations, and task information moved results more than time budget or model size. A verification tool changed behavior where a verification prompt barely did. TRACE found identical reruns flip 15-36% of task outcomes.

Test the scorer when a score moves. A score change can come from the scoring rule, not the agent. TRACE rescored unchanged trajectories after renaming tools: restoring the names at scoring time erased a 0.250 drop in a scripted agent.

Audit the simulated tools. Benchmarks grade what a simulated tool reports having done. An audit of 34 mutating tools in four benchmarks confirmed seven tool defects; in one clinical benchmark the tool claimed writes it never made, and the grader scored that claim as success. A defect like this is present on every rerun, so repeated runs never expose it.

Watch for saturation. Once scores plateau near the ceiling, the eval stops separating good from great, and further optimization tunes to its blind spots — the dynamic covered in does a high benchmark score predict production reliability?.

• Audit the eval before touching the agent. When a score looks bad, read failing transcripts by hand and check for ambiguous specs, wrong reference solutions, or a grader that missed a valid path.
• Start from 20-50 real failures. A small set from production catches the failures that matter faster than a large synthetic one.
• Report pass@k and pass^k. They answer "can it ever solve this" and "can I trust it every time"; one aggregate hides which you have.
• Match grader type to cadence. Code-based graders for anything deterministic, on every commit; model-based graders for tone, groundedness, and nuance; humans for periodic calibration of the automated graders.
• Grade the trajectory as well as the output. A lucky pass through a bad process is a risk the outcome score won't show.
• Rescore before you blame the agent. After a score change, rescore the unchanged trajectories under the old format; if the gap closes, the scorer moved, not the agent.
• Run each configuration several times and report the spread. At 54% run-to-run variance, a single run is mostly noise.
• Change behavior with tools, not prompt lines. Treat task information, budget, and tooling as explicit eval dimensions, and test self-verification as a tool or dedicated component.
• Check the tools against their contract. Probe each mutating tool in the eval and confirm the state it reports matches the state it wrote.
• Retire saturated evals. At ceiling, raise the difficulty or discount further gains.

• Debugging the wrong thing: changing the agent to chase a low score without checking the eval for ambiguous specs, wrong references, or grader bugs.
• One aggregate score: hiding whether failure is occasional (decent pass@k, bad pass^k) or total.
• Exact step matching: penalizing a correct result reached by a different valid path.
• No negative cases: overconfidence and unwarranted refusals never get caught.
• Trusting tool output as ground truth: a defective simulated tool makes the grader reward actions that never happened.
• Brittle scorers: a verifier that keys on tool names or output formatting turns a cosmetic change into a fake regression.
• Single-run comparisons: run-to-run variance read as a real difference.
• Misattribution: crediting the model for a change when task information, budget, or tooling also changed.
• Optimizing a saturated eval: tuning to its blind spots instead of real capability.

See agent evaluation, agent benchmarks for where public benchmarks diverge from your own eval, can you trust an LLM-as-judge score? for model-based grader failures, and does a high benchmark score predict production reliability? for reading a score as a production claim.