Agent Builder Foundations

Average success-rate improvement hides that skills cut both ways. Splitting outcomes into regressions (previously-passing tasks that now fail) and residual failures (tasks that never passed either way) shows the best skills mostly win by regressing less, not by solving more. Regressions cluster into three specific mechanisms, and the same three areas — procedural guidance, grounding, and verification — also explain most of what's left unsolved.

Treat a skill as a change to the agent's whole context, not a subroutine that only runs when invoked. A skill sitting in context can shift behavior on tasks that never call it, override how the agent reads its own inputs, and quietly turn off checks the agent would have run anyway. None of that shows up if you only measure "did the task pass," because a pass/fail count doesn't distinguish a task that was already broken from one your own change just broke.

The useful move is to split every outcome change into a regression (solved without the skill, failed with it) and a residual failure (failed either way). That separates "the skill didn't help" from "the skill broke something that worked". The Regression Tax study did this across nearly 6,000 runs and found three mechanisms behind most regressions:

• Skill description osmosis — the skill's presence in context changes behavior even on turns where it is never invoked.
• Grounding displacement — the prescribed procedure overrides how the agent reads its actual inputs, so it follows the recipe instead of the evidence.
• Verification displacement — the procedure supplies its own sense of "done" and suppresses checks the agent would otherwise run.

Residual failures show the same imbalance from the other side: skills over-invest in procedural guidance, the stage least often at fault, and under-support grounding and verification, where most remaining failures start.

• Score two numbers, not one. For every candidate skill, measure tasks newly solved and previously-passing tasks now failing separately — never collapse them into a single success-rate delta before shipping.
• Audit failures for the three named modes. When a task regresses, check whether the skill changed behavior on a turn it wasn't invoked on (osmosis), overrode input interpretation (grounding displacement), or suppressed an output check (verification displacement) before rewriting the procedure itself.
• Write grounding and verification steps as explicitly as the procedure. If a skill spells out steps but leaves "check your inputs" and "check your output" implicit, it's the shape most likely to displace exactly those checks.
• Re-test previously-passing tasks whenever a skill changes. A skill update that only gets evaluated against its target tasks will never surface a regression on tasks outside that set.

• Grading a skill by aggregate success-rate improvement, which lets regressions on previously-solved tasks hide behind gains on new ones.
• Assuming a skill only affects behavior when the agent actually invokes it, missing osmosis effects from the skill merely being present in context.
• Treating every regression as a procedure-writing problem and iterating on the steps, when the study finds grounding and verification gaps cause most of the remaining failures.
• Shipping a skill update without re-running the agent's previously-passing task set, so a new regression ships silently.

See tool use for the broader set of failure modes in connecting agents to real tools, and agent reliability for why fluent, confident-looking output doesn't imply the agent's checks are still running.