{"date":"2026-11-03","title":"Agent Builder's Playbook — Nov 3, 2026","generated_at":"2026-11-03T10:04:46Z","intro":["Agent harnesses, not models, decided outcomes this week: tool choice, hidden token spend and unchecked tests all moved results."],"card_count":6,"cards":[{"id":"pb-test-mcp-per-harness","kind":"source-backed","title":"Benchmark your MCP server through each agent harness, not just one model","area":"Tool use","problem":"The same model scored 82.0% through one coding agent and 65.4% through another on the same 21 MCP servers.","apply":"Run your MCP server's task suite through every harness your users run, and check which built-in tools the agent reaches for before your server's tools.","result":"You find harness-specific failures before users do, instead of blaming the model.","effort":"medium","source":"hackernews_ai","source_url":"https://m3.sineframe.com/blog/same-model-different-agent","source_sid":"b78023b26122d472","evidence":{"kind":"source-measured","note":"The source measured 82.0% vs 65.4% for Claude Haiku 5.5 across 21 servers and three trials per task."}},{"id":"pb-mustfail-revert-check","kind":"source-backed","title":"Revert the agent's source change and re-run its tests to prove they can fail","area":"Evals","problem":"A green suite from a coding agent can still be vacuous when it widens a mock or asserts only that a value is defined.","apply":"After the agent finishes, revert its source changes, keep its new tests unchanged, and re-run; treat any test that still passes as not testing the work.","result":"Tautological tests get flagged before merge instead of being trusted as proof.","effort":"low","source":"hackernews_ai","source_url":"https://github.com/greenlitbooks/mustfail","source_sid":"a53f6b2859b09ad7","evidence":{"kind":"editorial-inference","note":"The source describes the technique and failure modes; no measured outcome is claimed here."}},{"id":"pb-detect-repeated-tool-calls","kind":"source-backed","title":"Cap identical tool calls so a looping agent stops early","area":"Reliability","problem":"An agent re-ran the same successful SQL query 254 times, burning tokens and database load while appearing to work.","apply":"Hash each tool call's name and arguments per task, and stop or force a different action once the same call succeeds a few times in a row.","result":"Runaway loops end early instead of draining budget and hammering your data layer.","effort":"low","source":"hackernews_ai","source_url":"https://armincat.com/articles/ai-agent-executed-same-query-254-times","source_sid":"e79cc677e615b2d8","evidence":{"kind":"editorial-inference","note":"The incident report documents the loop; the guard's payoff is the editor's inference."}},{"id":"pb-audit-harness-background-tokens","kind":"source-backed","title":"Audit hidden harness calls that spend tokens outside your task","area":"Cost & latency","problem":"Harnesses make background model calls, such as session naming and status-line updates, that never show up in your task logs.","apply":"Log every model request your harness sends, including background ones, and disable or route the non-essential calls to a cheaper model.","result":"The author claims this exposes spend that does no task work, such as 15-20% of tokens on a status line.","effort":"low","source":"hackernews_ai","source_url":"https://armincat.com/articles/where-tokens-go","source_sid":"16f84d551cfc686c","evidence":{"kind":"source-claimed","note":"The author reports Claude Code spends 15-20% of tokens on a status line; this is a claim from one reading of the code."}},{"id":"pb-measure-cli-startup-overhead","kind":"source-backed","title":"Measure per-job startup cost before running a coding-agent CLI in fresh containers","area":"Cost & latency","problem":"A coding-agent CLI that feels fast in a terminal can be costly when every workflow job starts it in a fresh container.","apply":"Benchmark candidate CLIs on start-up time, time per turn, memory and image size in your own CI, then weigh that against task success separately.","result":"You pick a harness whose per-job overhead fits your pipeline instead of guessing.","effort":"medium","source":"hackernews_ai","source_url":"https://aictrl.dev/blog/headless-coding-agent-cli-benchmark","source_sid":"18df784100d9cc05","evidence":{"kind":"source-measured","note":"The source measured start-up, per-turn time, memory, image size and tokens for six CLIs, and says it did not measure task completion."}},{"id":"pb-pin-tool-versions","kind":"source-backed","title":"Re-validate tool definitions when a tool updates, not just at install","area":"Safety","problem":"Agent frameworks keep trusting a tool after a software update changes what it does.","apply":"Pin tool descriptions and schemas by hash, and require re-approval or a diff review whenever an MCP server or tool changes them.","result":"A silently modified tool can no longer inject new behavior into an agent that approved the old one.","effort":"medium","source":"arxiv_cs_ai","source_url":"http://arxiv.org/abs/2611.01976v1","source_sid":"3b426192f1bc75ae","evidence":{"kind":"editorial-inference","note":"The paper frames the tool update gap; the proposed control is the editor's inference."}}]}