Eval Engineering Skill: Build Evals From Repo Context and Traces
LangChain's new skill reads an agent's repo and production traces, proposes evals through user interviews, and outputs runnable Harbor test tasks.
AI Daily Recap
19 articles · 5 categories
The finishable daily brief
What happened in AI — Jul 22, 2026
Wednesday, Jul 22, 2026
19 articles · 5 categories
read top to bottom · then stop
In 30 seconds
- vLLM shipped production-scale serving support for Kimi K3 while Microsoft weighs swapping it into Copilot to save $600M on inference.
- The White House escalated accusations that Moonshot AI trained Kimi K3 on smuggled Nvidia chips and IP stolen from Anthropic.
- LangChain's new Eval Engineering Skill turns an agent's repo and traces into runnable evals automatically.
- Anthropic published the sandboxing architecture — filesystem, network, and execution limits — it uses to contain Claude across its own products.
- A wave of Show HN coding-agent tools launched: a self-hosted LLM router (Millwright), two native-Mac agent terminals (Rabbitty, Forkbench), and an autonomous eval-writing 'AI engineer' (Langy).
- Anthropic committed $200M to an Economic Futures Research Fund; OpenAI launched Presence, an enterprise voice/chat agent platform.
Moonshot's Kimi K3 forced a simultaneous economic and political reckoning today: vLLM shipped production-grade serving support, Microsoft is evaluating swapping ChatGPT and Claude out of Copilot to cut inference costs by $600M, and the White House escalated claims that the model was built on smuggled Nvidia chips and cloned Anthropic IP.
On the practice side, LangChain shipped a skill that generates evals straight from an agent's repo and production traces, and Anthropic published the containment architecture — filesystem, network, and execution limits — it uses to sandbox Claude across Web, Code, and Cowork.
Agent Engineering: Evals and Composition 3 items
Eval generation and agent architecture both moved from talk to shipped tooling: LangChain now automates eval creation from real traces, and a conference talk argues agents need versioned, composable "virtual tools" instead of ad hoc prompt chains.
3 Years of Graph Engineering with LangGraph
LangChain frames LangGraph's three years as validation that graph-based orchestration, not single-model prompting, is the durable pattern for reliable agents.
Presentation: From Copy-Paste to Composition: Building Agents Like Real Software
Jake Mannix argues agents should move past ad hoc "1970s BASIC" architectures toward an intermediate protocol layer of versioned, encapsulated "virtual tools."
Security & Containment 3 items
Agent containment moved from research topic to operational priority: Anthropic detailed the sandboxing limits it runs Claude under, a Chinese lab's model was reportedly hacked after copying OpenAI outputs, and analysts flagged cybersecurity as a rising theme across this week's AI coverage.
Anthropic Details How It Contains Claude Across Web, Code, and Cowork
Anthropic argues agent safety depends on deterministic limits on an agent's filesystem, network, and execution environment, not just model-level alignment.
China's Zhipu AI model contains hack after OpenAI models go rogue - South China Morning Post
A report says Zhipu AI's model was compromised after ingesting outputs from misbehaving OpenAI models, an early example of contamination risk from cross-model data reuse.
[AINews] AI Cybersecurity becomes top of mind
A roundup notes a cluster of new cybersecurity-focused AI headlines this week, signaling security is becoming a recurring editorial theme rather than a one-off story.
Kimi K3: The Model That Reshuffled the Market 5 items
Moonshot's Kimi K3 forced simultaneous engineering, business, and political responses today: vLLM shipped optimized production serving, Microsoft is weighing a swap into Copilot to cut costs, and the White House escalated IP-theft and chip-smuggling accusations against Moonshot.
A Preview of Production-Scale Kimi K3 Support on vLLM
vLLM previewed production-scale Kimi K3 serving, including KDA-aware prefix caching, fused kernels, optimized MXFP4 MoE, and multimodal support on both NVIDIA and AMD paths.
Microsoft considers replacing ChatGPT and Claude with China's Kimi K3 to save $600 million - moneywise.com
Microsoft is reportedly evaluating Kimi K3 as a Copilot backend to cut roughly $600M in inference costs versus its current OpenAI and Anthropic model mix.
Open models recap: more on Kimi K3, Qwen 3.8, Xi's WAIC speech, distillation, the open-closed gap, and what's next
Interconnects' recap digs into whether Kimi K3 and Qwen 3.8 were distilled from closed frontier models, and what that implies for how fast the open-closed performance gap keeps closing.
White House accused China's Moonshot AI of accessing banned Nvidia chips and stealing from Anthropic - qz.com
A White House official accused Moonshot AI of training Kimi K3 using export-controlled Nvidia chips and technology taken from Anthropic, escalating the dispute beyond generic IP claims.
OpenAI President Says Kimi K3 "Pretty Good," Unsure If Distilled - Bloomberg News - TradingView
OpenAI's president publicly called Kimi K3 "pretty good" while declining to confirm whether it was distilled from a closed frontier model, a notably measured reaction from a direct competitor.
Coding-Agent Tooling: Show HN Roundup 5 items
A cluster of new tools targets how builders run and pay for coding agents: a self-hosted router to cut per-call cost, two native-Mac control surfaces for running multiple agents at once, and a fresh breakdown of what Copilot's usage-based billing actually buys versus raw API access.
Show HN: Millwright – Rust-based, self-hosted LLM router
Millwright is a self-hosted, Rust-based LLM router built for cost control and transparency, a response to hosted routers proliferating (and OpenRouter's possible acquisition) without an open alternative.
Copilot vs. raw API access: What are you actually paying for?
GitHub breaks down what Copilot's now-listed-API-rate billing buys beyond raw model access: the coding workflow, policy controls, and harness engineering wrapped around the calls.
Rabbitty – a native Mac terminal for running AI coding agents in parallel
Rabbitty is a native macOS terminal built specifically to run and manage multiple AI coding agents side by side.
Show HN: Forkbench – a native-Mac control room for running CLI coding agent
Forkbench is another native-Mac control surface for supervising CLI coding agents, part of a fast-forming category of agent-fleet management tools.
Show HN: Langy, an automated AI engineer (we gave it a robot body) [video]
Langy reads a platform's production traces, writes Scenario tests and evaluations for problems it finds, and opens a pull request against the repo on its own.
Platform Bets: Enterprise Agents and Research Funding 3 items
The frontier labs made three large forward-looking commitments: OpenAI launched an enterprise voice/chat agent platform, Anthropic funded external economic research on AI's labor impact, and Google earmarked compute credits for scientific-discovery research.
Introducing OpenAI Presence
OpenAI launched Presence, positioned as a proven enterprise agent platform for deploying trusted voice and chat agents across customer and internal workflows.
Supporting ambitious external research through the Anthropic Economic Futures Research Fund
Anthropic committed $200 million to fund external research on AI's economic effects, aiming to get independent data ahead of policy debates rather than after them.
Accelerating the frontiers of scientific discovery: Google's $40M commitment to the Genesis Mission
Google committed $40M in AI tokens and compute credits to the Genesis Mission, a push to apply frontier models directly to open scientific-discovery problems.
You are caught up for this edition