Agent Know-How

The substrate churns across several layers, and each is a drift source:

• Frameworks ship fast and regress: LangGraph 1.2.6 had to fix nested subgraphs inheriting the parent checkpoint namespace — a regression introduced two releases earlier in 1.2.3 — meaning anyone who upgraded into that window silently got broken checkpointing without touching their own code.
• Agent SDKs move almost daily: the Claude Agent SDK for Python ships releases whose entire changelog is "updated the bundled Claude CLI," so the executable your agent runs on changes underneath a patch-level dependency bump. That cadence has not let up: the most recent week saw the SDK roll from 0.2.115 through 0.2.120, six releases in a row advancing only the vendored CLI (2.1.206 → 2.1.211) — except one of them wasn't purely cosmetic: the 0.2.116 bump carried a CLI fix so Claude Code honors project-scoped permission grants in checkout directories, a real permission-behavior change riding on what its own changelog entry made look like just another CLI version bump. The pattern repeated two days later at larger scale: 0.2.122's changelog is again just "updated bundled Claude CLI," this time forwarding claude-code v2.1.214 — a release whose own notes list five distinct permission-check bypass fixes (a Windows PowerShell 5.1 check bypass, docker commands with daemon-redirect flags escaping approval, dir/** allow-rules over-matching outside their intended directory, long commands auto-approving past a 10,000-character threshold, and zsh variable-subscript mishandling in Bash checks). The one-line-changelog pattern hasn't slowed since: 0.2.123 forwards claude-code v2.1.215 with the same single bullet ("updated bundled Claude CLI"), and it kept recurring three releases later — 0.2.125 again reads only "updated bundled Claude CLI," this time forwarding v2.1.217 — so a team tracking only the SDK's own version number still has to open the CLI's own release notes to know what actually changed underneath it, every single release, not just occasionally. The next two releases broke from that pure-cosmetic pattern in opposite, equally consequential directions: v0.2.126 shipped real new API surface instead of just a CLI bump — ResultMessage.terminal_reason now surfaces why the query loop ended ("completed", "max_turns", "aborted_streaming", ...) and ResultMessage.model_usage gives typed per-model token/cost usage, both load-bearing for retry and cost logic built on top of the SDK — while v0.2.127 paired a genuine bug fix (query() no longer closes stdin on the first result frame while background tasks are still in flight) with, again, a bundled-CLI bump, this time to v2.1.219. A team that pins only the SDK version and skims changelogs for keywords can miss exactly this kind of drift. The pure-cosmetic pattern then resumed at pace: v0.2.130 and v0.2.131 are each again a single "updated bundled Claude CLI" line (forwarding v2.1.222, then v2.1.223) with no other changes disclosed — but the CLI release riding underneath one of those bumps, v2.1.221, is not cosmetic at all: it fixes a Bash permission-check bypass where zsh could execute hidden commands inside [[ ]] regex conditionals, a Windows PowerShell permission check mishandling quoted paths, and adds a mode: "mask" sandbox setting so sandboxed commands read a sentinel credential file while a proxy substitutes the real value only on egress — the same "permission-bypass fixes hidden inside a one-line SDK changelog" shape the v2.1.214 case already established, recurring on a different CLI version. A fourth wave, three weeks later, is the largest yet: four more one-line "updated bundled Claude CLI" releases (v0.2.135, v0.2.136, v0.2.138, v0.2.139) forward CLI v2.1.227, v2.1.228, v2.1.232, and v2.1.233 respectively, and three of those four CLI releases carry undisclosed security fixes — v2.1.227 alone fixes four issues (a crafted-command Bash permission-check bypass, tab/invisible-Unicode characters that hid parts of a command from the approval dialog, a workflow-sandbox escape via dynamic import(), and an agent-definition bypassPermissions mode that ignored an org's disable policy), v2.1.232 fixes a PowerShell bypass (variable-writing parameters silently overwriting $PSDefaultParameterValues) and a Windows Git Bash bypass (Cygwin-style symlinks evading path validation), and v2.1.233 closes an NTLM credential-leak vector where a Windows NT \??\ device-prefix path bypassed UNC path validation. The one release in between, v2.1.228, hardens skills synced from claude.ai so they no longer shadow local commands or MCP prompts and can no longer run ! shell commands or expand @ file references from their body — a supply-chain-shaped fix for synced, not locally-authored, content. Four waves of "permission-bypass fixes hidden inside a one-line SDK changelog" in two months is no longer an anomaly in this dependency's release shape; it's the default one. A fifth wave, a week later, forwards on two different axes at once: v0.2.143 and v0.2.144 are each again a single "updated bundled Claude CLI" line, forwarding claude-code v2.1.238 and v2.1.246 respectively, and v2.1.238 continues the specific recurring theme the v2.1.221 case established — Claude Code "improved Bash tool permission checking for zsh-specific syntax in shell conditionals," another patch to the same class of zsh-conditional loophole rather than a one-off — while v2.1.246 pairs a real permission-check bypass fix (Bash commands with a malformed dangling && or || operator now always require approval, closing a path that could previously skip the prompt) with a credential-leak fix (telemetry and metrics requests no longer carry the API key configured for a third-party ANTHROPIC_BASE_URL gateway to the wrong host) and a sandbox fix (the command sandbox's filesystem configuration now respects --setting-sources), plus a startup warning that Bash allow rules with a wildcard before the subcommand (e.g. Bash(git * main)) also match options inserted before it — a permission-rule-matching pitfall surfaced for operators rather than silently closed. v0.2.145, forwarding v2.1.247, breaks the pattern in the opposite direction again, the same way v0.2.126 did months earlier: its own SDK changelog is still just "updated bundled Claude CLI," but the CLI underneath ships real new capability — a SendFeedback tool and a /claude-api cost-optimize skill — with no hint of either in the SDK's own release notes. The regress-then-revert shape isn't unique to the SDK's forwarding layer either: claude-code v2.1.270 fixed a v2.1.269 regression where read-only git commands in Bash started unexpectedly asking for permission again after a session had been running for a while, and the same shape repeated a week later one level up — the Claude Agent SDK's v0.2.155 forwarded claude-code v2.1.275, which broke every request behind an ANTHROPIC_BASE_URL proxy or gateway with a 400 error on an unrecognized advisor_20260301 input tag, and v0.2.156's changelog, still just "updated bundled Claude CLI," was itself the fix, forwarding v2.1.276 — so the one-line SDK changelog carried both the break and the repair without describing either. Two releases before that pair, v0.2.153 broke from the cosmetic pattern with real new API surface of its own: a snapshot field on SystemPromptPreset and a new SystemPromptCustom typed dict, letting a session snapshot its system prompt, shipped alongside the usual CLI bump (to v2.1.273).
• Reproducing a drift-caused failure is a distinct problem from detecting one: an LLM agent's failures are hard to reproduce because inference isn't bitwise-deterministic, tools read state that has since moved on, and a failure can depend on a long multi-step trajectory — the Chronicle paper's answer is cut-point replay, resuming a regression test from a specific point in a saved trajectory instead of re-running the whole session, so a regression can be isolated without re-triggering the same volatile conditions that made it hard to reproduce in the first place.
• Community tooling is starting to treat this obstacle as its own category: Drift, an open-source, intent-driven versioning tool for AI coding agents, frames exactly the problem this bullet documents — that an agent's behavior can shift between ordinary-looking releases — as something a team should version and diff explicitly, rather than discover after the fact from a changelog line that undersells what changed.
• Models get deprecated out from under running agents — Claude Code now emits a warning when the requested model is deprecated, making model-upgrade drift an explicit, surfaced signal rather than a silent behavior change — and the same release hardened auto-mode safety (blocking destructive git commands), a reminder that the harness's *defaults* drift too. Claude Code v2.1.219 makes the model-upgrade case concrete rather than hypothetical: it added Claude Opus 5 (claude-opus-5) as the new default Opus model — 1M context, fast mode at $10/$50 per Mtok — so any code or agent that referenced "the default Opus model" now gets a different model, a larger context window, and different pricing without a single line of its own code changing.
• Serving runtimes drift in performance and output: vLLM v0.23.0 is another "hardening and optimization pass" on DeepSeek-V4 across backends, the kind of change that can move latency, throughput, and sampling behavior without a model swap, and the drift can be outright breaking, not just behavioral — Triton Inference Server's 2.70.0 release drops Windows support entirely and changes how its Python client handles BF16 (now requiring ml_dtypes), so a runtime bump can remove a deployment target or break client code that never touched the model. A controlled study puts a number on how much of that drift is the backend alone, isolated from the model: crossing three instruction-tuned models against five inference frameworks (HuggingFace, vLLM, Ollama, and others) and six benchmarks under deterministic, sampling-noise-free decoding, the serving backend explains roughly 39% of the score variance a practitioner sees out of the box — meaning "which inference framework and version produced this number" belongs next to "which model" as a variable a team pins and discloses, not one it can treat as non-influential plumbing (see agent evaluation for the same finding from the benchmarking-pipeline side).
• Coding-agent CLIs regress and roll back like any other dependency: OpenAI's Codex CLI shipped a prompting regression in its Guardian auto-review behavior, then reverted it two releases later — 0.144.2 restored the prior policy, request format, and tool behavior, followed by a version-only 0.144.3 with no further changes — the same "patch-level bump changes behavior" pattern the Claude Agent SDK bullet above describes, this time inside the auto-review policy an agent enforces rather than the CLI binary underneath it. The one-line-changelog pattern isn't Anthropic-specific either: Codex 0.144.6's changelog reads as a routine "refreshed bundled instructions" note for its GPT-5.6 Sol, Terra, and Luna models, but folded into that refresh was a correction to their context windows (272,000 tokens) — model metadata that routing and token-budget code silently depends on, changing in a point release with no separate callout. The same CLI's auto-review policy drifted again, in the opposite direction from the earlier regression-and-revert: 0.146.1 backported "safer automatic-review defaults for cyber-capable models," tightening the guardrail behavior an agent enforces on models flagged for cyber capability — a policy change delivered as a routine bugfix release, the same shape as the earlier Guardian regression but reflecting the same cyber-eval-incident pressure now showing up on the prompt injection and agent evaluation pages.

The field is starting to give operators levers — LangGraph's CLI now supports declaring *compatible API version ranges* — but the default posture is still "track latest," which is exactly how drift gets in.

The migration itself, not just detecting drift, is a named practitioner topic now: Google Cloud published lessons learned from accelerating foundation-model upgrades across engineering teams, reinforcing that the upgrade path — not just the deprecation warning — is where the drift this page tracks actually has to be managed (see version pinning for the specific migration case this evidence also grounds).

Silent prompt and inference-parameter changes drift quality too, not just SDK/CLI code — Anthropic's own postmortem on a six-week run of Claude Code quality complaints (March 4–April 20, 2026) traces it to three separate, un-flagged changes on Anthropic's own side, none of which touched a pinned model or SDK version: the default reasoning effort quietly dropped from high to medium to cut latency (reverted after user complaints, with Opus 4.7 now defaulting to xhigh); a prompt-caching bug used the clear_thinking_20251015 header incorrectly and cleared reasoning state on *every* turn instead of once, making Claude look "forgetful and repetitive" until a v2.1.101 fix — and it stayed hidden because two unrelated changes under test at the same time masked the symptom, surfacing only when a newer model (Opus 4.7) reviewed the older model's own session logs in retrospective Code Review; and a verbosity-limiting system-prompt addition ("keep text between tool calls to ≤25 words") cost a measured 3% on broader evals before being reverted the same week it shipped. The drift arrived entirely inside the vendor's own defaults and prompts — nothing a team pinning its model or SDK version would have caught. Anthropic's remediation doubles as the regression-gate discipline this page already argues for, applied to the vendor's own changes: per-model evals and ablation testing before a system-prompt or default change ships, soak periods and gradual rollouts instead of an instant flip, and expanded internal dogfooding on public builds rather than an internal-only build that can drift from what users actually run.

claude-code v2.1.275 broke every request behind an ANTHROPIC_BASE_URL proxy or gateway with a 400 error, fixed one release later in v2.1.276 — and the Claude Agent SDK's one-line "updated bundled Claude CLI" changelog carried both the regression (v0.2.155) and the fix (v0.2.156) without describing either (see State of the art above).

This is the obstacle that breaks an agent you already shipped, on a day you didn't deploy. You own the agent but rent the substrate, and its release cadence isn't yours — a framework patch can reintroduce a regression, an SDK bump can swap the executable, and a model deprecation can change behavior or pull the model entirely. The discipline is to treat the model, SDK, and serving runtime as pinned, version-controlled dependencies with a regression gate (see version pinning and agent benchmarks) — staged, tested upgrades, not a rolling "latest." Drift trades against freshness: the newest model or framework is also the one most likely to move under you.