An agent is built on a substrate you don't control and that moves faster than your app: the model gets upgraded or deprecated, the agent SDK and framework ship several releases a week, and the serving runtime changes behavior. Every bump can silently change what the agent does, between two deploys where your own code never changed.
Every layer under the agent is a drift source. Frameworks ship regressions and fix them two releases later. Coding-agent CLIs change permission checks, auto-review policy, and model metadata in point releases. Serving runtimes move latency and sampling, and the backend alone explains a large share of benchmark variance.
Changelogs are not a reliable signal. The clearest pattern is the Claude Agent SDK: most releases say only "updated bundled CLI", while the forwarded CLI carries permission-bypass fixes, credential-leak fixes, new tools, or a break that took down every request behind a gateway. The same shape appears in Codex point releases that quietly change context windows and review defaults.
Pinning is necessary but not sufficient. Anthropic's own Claude Code quality postmortem traced weeks of complaints to vendor-side changes in default reasoning effort, a caching bug, and a system-prompt line. None touched a model or SDK version a customer could pin. Default-model swaps (a new default Opus) do the same to anyone who referenced "the default".
The consensus discipline is to treat model, SDK, framework, and runtime as pinned dependencies behind a regression gate: declared compatible version ranges, staged upgrades, and evals before rollout. Vendors now apply the same gate to their own prompt and default changes.
The open problem is reproduction. Non-deterministic inference and moving tool state make a drift-caused failure hard to replay. Record-and-replay from a cut point in a saved trajectory is the newest answer, but tooling is early and "track latest" remains the ecosystem default.
Why it matters for platform engineers
This is the obstacle that breaks an agent you already shipped, on a day you didn't deploy. You own the agent but rent the substrate, and its release cadence isn't yours.
Treat the model, SDK, bundled CLI, and serving runtime as pinned, version-controlled dependencies with a regression gate (see version pinning and agent benchmarks). Read the forwarded component's release notes, not just the wrapper's, because security-relevant permission changes ride there. Drift trades against freshness: the newest model or framework is also the one most likely to move under you.
Developments 29
SDK bumps that hide what changed underneath
10 entries · latest Sep 18, 2026
Claude Agent SDK releases routinely say only "updated bundled CLI" while the forwarded CLI carries permission-bypass fixes, new tools, or breaking changes; the SDK changelog alone is not a reliable signal.
SDK v0.2.153 adds a snapshot field to SystemPromptPreset and a new SystemPromptCustom typed dict. With snapshot: True, a session keeps the system prompt recorded on its first request, improving prompt caching across resumed sessions; with False, the prompt is rebuilt every request. It requires CLI 2.1.257+ and ships with a bump to v2.1.273.
A resumed session can now run on an older system prompt than a fresh one, which matters when comparing behavior across sessions.
SDK v0.2.145's changelog is again just "Updated bundled Claude CLI", to v2.1.247. That CLI release ships new capability: a SendFeedback tool and a /claude-api cost-optimize skill.
The hidden change can be an addition as well as a fix. Either way, the agent's available tools changed without the SDK saying so.
SDK v0.2.143 and v0.2.144 forward claude-code v2.1.238 and v2.1.246 with single-line changelogs.
v2.1.238 improves Bash permission checks for zsh syntax in shell conditionals: the same class of loophole as v2.1.221, patched again.
v2.1.246 requires approval for commands with a dangling && or ||, stops telemetry sending a third-party ANTHROPIC_BASE_URL gateway's API key to the wrong host, and makes the sandbox respect --setting-sources. It also warns that rules like Bash(git * main) match options inserted before the subcommand.
SDK v0.2.135, v0.2.136, v0.2.138, and v0.2.139 forward CLI v2.1.227, v2.1.228, v2.1.232, and v2.1.233. Per the CLI notes:
v2.1.227: a crafted-command Bash bypass, invisible Unicode hiding parts of a command from approval, a workflow-sandbox escape via dynamic import(), and bypassPermissions ignoring an org's disable policy.
v2.1.228: synced claude.ai skills can no longer shadow local commands or run ! shell commands.
v2.1.232: PowerShell and Git Bash symlink bypasses.
v2.1.233: an NTLM credential leak via \??\ paths.
Four waves in two months make this the dependency's normal release shape.
SDK v0.2.130 and v0.2.131 again read only "Updated bundled Claude CLI" (to v2.1.222, then v2.1.223). The CLI release just before them, v2.1.221, fixes a Bash permission-check bypass where zsh could run hidden commands inside [[ ]] regex conditionals, and a Windows PowerShell check mishandling quoted paths.
It also adds mode: "mask" for sandbox credential files: sandboxed commands read a sentinel copy while the sandbox proxy substitutes the real value on egress. On macOS, masking falls back to deny.
v0.2.126 adds ResultMessage.terminal_reason (why the loop ended: completed, max_turns, aborted_streaming, ...) and types model_usage per model. Both are load-bearing for retry and cost logic.
v0.2.127 stops query() closing stdin on the first result frame while background tasks run. Before, SDK-MCP tool calls from background subagents failed with "Stream closed" and silently bypassed PreToolUse hooks. It also bumps the CLI to v2.1.219.
Real API surface and a hook-bypass fix ship in the same stream as cosmetic bumps.
The Claude Agent SDK rolled from v0.2.115 to v0.2.120 in about a week, advancing the bundled CLI from 2.1.206 to 2.1.211. Most entries say only "Updated bundled Claude CLI".
v0.2.116 was not cosmetic: it fixed CI workspace trust so Claude Code honors project-scoped permission grants in checkout directories. v0.2.117 also escaped untrusted fields in the repo's Slack notification workflow.
A permission-behavior change arrived looking like one more version bump. Keyword-skimming the SDK changelog would miss it.
claude-agent-sdk-python v0.2.106 and v0.2.110 each list a single change: "Updated bundled Claude CLI", to 2.1.185 and 2.1.191. One CLI release in that span, v2.1.190, describes itself only as "Bug fixes and reliability improvements".
A patch-level SDK bump swaps the executable your agent runs on, and neither changelog says what behavior moved. Treat the bundled CLI version as a dependency you track separately.
Regressions shipped and rolled back in point releases
4 entries · latest Sep 18, 2026
Frameworks, coding-agent CLIs, and SDKs regularly ship a regression and revert it one or two releases later, so anyone who upgraded in the window got broken behavior without changing code.
claude-code v2.1.275 made every request fail with a 400 on an unrecognized advisor_20260301 input tag whenever ANTHROPIC_BASE_URL pointed at a proxy or gateway. v2.1.276 fixed it.
The Claude Agent SDK forwarded both: v0.2.155 carried the break and v0.2.156 the fix, each with the one-line "Updated bundled Claude CLI" changelog. Teams routing through an LLM gateway, the setup platform teams favor, were the ones broken.
claude-code v2.1.270 fixed read-only git commands in Bash asking for permission after a session had been running a while, a regression introduced in v2.1.269.
Permission behavior regressed and recovered across two adjacent patch releases. In unattended runs, an unexpected prompt is a stall, not an inconvenience.
Codex 0.144.2 restored the previous Guardian auto-review policy, request format, and tool behavior after rolling back a prompting regression. 0.144.3 followed as a version-only release with no changes.
The drift sat inside the auto-review policy the agent enforces, not the model or the binary. A patch-level bump changed safety behavior twice in two releases.
LangGraph 1.2.6 fixed nested subgraphs inheriting the parent checkpoint_ns, a regression introduced in 1.2.3. It also cancels running subgraphs on v3 stream abort.
Anyone who upgraded into the 1.2.3-1.2.5 window got broken checkpointing without touching their own code. Framework patch releases need the same regression gate as model upgrades.
Models, defaults, and prompts that change under you
6 entries · latest Sep 2, 2026
Default models, reasoning effort, system prompts, context-window metadata, and review policies change on the vendor side, often silently, and pinning a version does not catch them.
Anthropic's postmortem on complaints from March 4 to April 20, 2026 finds three un-flagged changes, none touching a pinned model or SDK version:
default reasoning effort cut from high to medium for latency (reverted)
a caching bug misusing clear_thinking_20251015 that cleared reasoning every turn, fixed in v2.1.101 and masked by two concurrent changes
a "≤25 words between tool calls" prompt line that cost 3% on broader evals
Remediation: per-model evals and ablations before prompt or default changes, soak periods, gradual rollouts, and dogfooding public builds. Pinning would not have caught any of it.
Codex 0.146.1 backports safer automatic-review defaults for cyber-capable models and explains permission changes in the terminal.
Guardrail behavior the agent enforces changed in a routine bugfix release, the same delivery shape as the earlier Guardian regression but in the tightening direction.
claude-code v2.1.219 adds Claude Opus 5 (claude-opus-5) as the new default Opus model: 1M context, fast mode at $10/$50 per Mtok.
Anything that referenced "the default Opus model" now gets a different model, a larger context window, and different pricing without a code change. Pin explicit model IDs where behavior or cost matters.
Codex 0.144.6 reads as a routine refresh of bundled instructions for GPT-5.6 Sol, Terra, and Luna, but it also corrects their context windows to 272,000 tokens.
Routing and token-budget code depends on that metadata, and it changed in a point release without a separate callout. The one-line-changelog problem is not specific to one vendor.
Google Cloud's lessons on accelerating foundation-model upgrades start from the observation that moving to a new model or checkpoint (for example, to Gemini 3.5) usually means months of manual testing to prove quality and evaluate new responses.
The migration path, not the deprecation notice, is where drift has to be managed. See version pinning.
claude-code v2.1.183 warns when the requested model is deprecated or automatically updated to a newer one, on stderr in print mode and for models set in agent frontmatter. The same release blocks destructive git commands and terraform/pulumi/cdk destroy in auto mode unless asked.
A silent model swap becomes a visible signal. The harness's safety defaults also moved in the same release.
Inference-runtime upgrades move latency, sampling, and even supported platforms, and the backend alone accounts for a large share of observed benchmark variance.
A fully crossed study (three instruction-tuned models × five inference frameworks such as HuggingFace, vLLM, and Ollama × six benchmarks × four generation modes) finds that the serving backend explains roughly 39% of the score variance a practitioner sees out of the box, even under deterministic decoding.
Framework name and version belong next to the model ID as variables you pin and disclose. See agent evaluation.
Triton Inference Server 2.70.0 (NGC 26.06) is a breaking release: it removes the Windows server build, and its Python client's BF16 handling now requires ml_dtypes.
Runtime drift can be outright breaking, not only behavioral: a bump can remove a deployment target or break client code that never touched the model.
vLLM v0.23.0 (408 commits from 200 contributors) gives DeepSeek-V4 another hardening and optimization pass: sparse MLA metadata decoupled from V3.2, a TRTLLM-gen attention kernel, EPLB for the Mega-MoE, and selective prefix-cache retention for sliding-window KV cache.
Changes like these can move latency, throughput, and sampling behavior with no model swap. A serving-runtime upgrade belongs behind the same regression gate.
Version ranges, intent-based versioning, cross-language tests, and cut-point replay are emerging, but the default posture across the ecosystem is still "track latest".
Modal made Endpoint Candidates generally available: a staged alternative to a Dedicated Endpoint (a catalog model or your own weights) that receives a sampled copy of production requests while production still answers users. Each Candidate has its own URL, so existing evals run against it, and the dashboard shows latency and throughput. Promotion starts a rollout at the original endpoint URL.
Mirroring is best-effort: each mirror gets a bounded buffer in a Pingora-based proxy, and a slow or failing Candidate is cancelled rather than slowing production. Mirrored compute is billed at standard rates.
For builders: a checkpoint or model swap can be tested on real traffic first, but quality still needs your own evals.
Agent failures are hard to reproduce: inference is not bitwise deterministic, tools read changed state, and re-runs rarely repeat a long trajectory. Chronicle records a run at its non-deterministic boundaries as immutable envelopes, and its cut-point replay resumes from a chosen point in the saved trajectory to test a code change against it.
Reproducing a drift-caused failure is a separate problem from detecting one. Replay isolates the regression without re-triggering the volatile conditions.
Drift is an open-source, intent-driven versioning tool for AI coding agents.
It frames behavior shifting between ordinary-looking releases as something to version and diff explicitly, rather than discover later from a changelog line that undersells what changed.
deptrust checks package versions for known vulnerabilities across npm, PyPI, crates.io, Go modules, Maven, GitHub Actions, and more, as a local CLI and an MCP server calling public registry and OSV APIs directly. Its author built it because coding agents kept suggesting outdated or vulnerable versions.
An agent's knowledge of the ecosystem drifts out of date. A verification tool in the loop corrects it at suggestion time.
langgraph-cli 0.4.30 adds support for compatible API version ranges.
It is an operator lever against drift: declare what you have tested instead of tracking latest. The default posture across the ecosystem is still "track latest", which is how drift gets in.