If you ship a procedural skill (a step-by-step playbook injected into an agent's context) and only track average task-success rate, you can ship a net regression without seeing it. A skill that fixes ten tasks and quietly breaks eight tasks the agent used to pass looks like a solid win on the aggregate number, but eight users just watched something that worked stop working.
"},{"heading":"Short answer","html":"Average success-rate improvement hides that skills cut both ways. Splitting outcomes into regressions (previously-passing tasks that now fail) and residual failures (tasks that never passed either way) shows the best skills mostly win by regressing less, not by solving more. Regressions cluster into three specific mechanisms, and the same three areas — procedural guidance, grounding, and verification — also explain most of what's left unsolved.
"},{"heading":"Builder model","html":"Treat a skill as a change to the agent's whole context, not a subroutine that only runs when invoked. A skill sitting in context can shift behavior on tasks that never call it, override how the agent reads its own inputs, and quietly turn off checks the agent would have run anyway. None of that shows up if you only measure "did the task pass," because a pass/fail count doesn't distinguish a task that was already broken from one your own change just broke.
"},{"heading":"Mechanism","html":"The study runs agents with and without a candidate skill across nearly 6,000 tasks, two office-automation benchmarks, and three harness stacks, then buckets every outcome change into one of two categories: a regression (solved without the skill, failed with it) or a residual failure (failed either way). This decomposition is the whole point — it separates "the skill didn't help" from "the skill actively broke something that worked."
\nThree mechanisms explain most regressions:
\nLooking at residual failures (tasks that still fail with the skill) turns up a matching imbalance rather than a different problem: existing skills over-invest in procedural guidance, the stage the study finds is least often the actual cause of failure, while under-supporting grounding and verification, the stages responsible for most of what's left unsolved.
"},{"heading":"Evidence","html":"Benchmark/result-backed: nearly 6,000 runs across two office-automation benchmarks and three harness stacks, with outcomes decomposed into regressions vs. residual failures rather than reported as a single success-rate delta. The paper reports the direction and mechanism of the effect (regressing-less beats gaining-more among top skills; three named regression causes) without publishing a specific percentage for how much of the improvement each mechanism explains — treat the mechanism finding as established and any percentage as unstated by the source.
"},{"heading":"How to apply","html":"See tool use for the broader set of failure modes in connecting agents to real tools, and agent reliability for why fluent, confident-looking output doesn't imply the agent's checks are still running.
"}],"evidence":[{"id":"regression-tax-2026-skills","kind":"benchmark-result","tier":"benchmark/result-backed","title":"The Regression Tax: Decomposing Why Skills Help and Hurt LLM Agents","note":"Compares agents with and without a procedural skill across nearly 6,000 runs spanning two office-automation benchmarks and three model harness stacks. Splits outcomes into a regression (a task the agent solved without the skill but fails once the skill is added) versus a residual failure (a task that fails both with and without the skill). Finds the best-performing skills win primarily by regressing less, not by gaining more, and identifies three regression causes: skill description osmosis (the skill changes behavior just by being present in context, even when never invoked), grounding displacement (the skill's prescribed procedure overrides how the agent reads its inputs), and verification displacement (the procedure suppresses checks the agent would otherwise run on its own outputs). Analyzing persistent (residual) failures finds the same pattern in reverse: existing skills overemphasize procedural guidance, the stage least often responsible for failure, while under-supporting grounding and verification, the stages that cause most remaining errors.","url":"http://arxiv.org/abs/2607.22520v1"},{"id":"story-89a606f362d88b4e-regression-tax","kind":"story","tier":"source story","title":"The Regression Tax: Decomposing Why Skills Help and Hurt LLM Agents","note":"","sid":"89a606f362d88b4e"}],"related_topics":[{"slug":"tool-use","title":"Agents reach the outside world through fragile, ad-hoc integrations"},{"slug":"agent-reliability","title":"Agents give fluent, confident-looking output even when it's wrong"}],"related_playbook_cards":["pb-skills-regression-tax"],"related_storylines":[],"covers_evidence":["regression-tax-2026-skills","story-89a606f362d88b4e-regression-tax"]}