Obstacle
Agents give fluent, confident-looking output even when it's wrong
An agent can hallucinate a fact, skip a step, or misuse a tool and still return a fluent, confident-looking answer — nothing about the output itself signals that it's wrong.…
Agent Know-How
Agent engineering
The essential know-how for production-grade agents
Everything here orbits one job: keeping an agent alive in production. Each planet is a problem you will eventually hit — memory, cost, evals, security — linked to the solutions the field actually uses. Every claim is grounded in source articles already in the feed — one shared structure, no hype.
13 obstacles·11 solutions·12 areas
Orbit viewbright satellite = unread · dim = read · ✦ ping = updated recently
Every planet is a problem you eventually hit in production; the satellites circling it are the pages that show how the field handles it. Tour planet by planet with ‹ › · drag to orbit · scroll or pinch to zoom · click a planet to see what lives there. What you read dims — remembered only in this browser.
The same universe, as a list
Reliability & correctness
Memory & context
Obstacle
Agents forget across steps and sessions
An agent's working memory is its context window, which is finite and resets between runs. On long-horizon tasks it forgets earlier steps, repeats work, and loses the user's…
Planning & reasoning
Obstacle
Agents plan multi-step work badly — they loop, stall, or skip steps
Give an agent a goal that takes ten steps and it will often take the wrong ones: charge ahead on an ambiguous request instead of asking, decompose the task into a plan that…
Tool use & interop
Obstacle
Agents reach the outside world through fragile, ad-hoc integrations
An agent is only as useful as the tools it can call, but every integration has historically been bespoke: hand-written wrappers around REST APIs, brittle schemas the model…
Grounding & knowledge
Obstacle
An agent's answer is only as good as what it retrieved — and whether it can prove it
A fluent agent answer isn't the same as a grounded one: the model will answer past what it actually retrieved unless the retrieval was current, the right slice, and cheap enough…
Evaluation
Obstacle
Measuring whether an agent actually worked is hard
A chatbot is graded on its final answer; an agent has to be graded on what it *did* — the multi-step trajectory of tool calls, retries, and decisions that led there. Outputs are…
Multi-agent coordination
Obstacle
Coordinating multiple agents adds more failure than capability
Splitting a job across several agents promises specialization and parallelism, but every handoff is a lossy interface and each added agent multiplies the ways the system can…
Cost
Obstacle
Agent token costs are unpredictable and easily run away
A chatbot turn costs a predictable number of tokens; an agent can loop, re-read its whole context every step, spawn sub-agents, and call a model to grade its own work — so the…
Obstacle
Proving agent ROI and measuring cost efficiency is hard
Calculating the true return on investment (ROI) for agent systems is blocked by the difficulty of measuring time-savings, tracking per-task token usage, and accounting for hidden…
Latency & throughput
Obstacle
Agent loops multiply per-call latency into slow, expensive runs
A chatbot waits on one model call; an agent waits on *many*, in sequence — plan, call a tool, read the result, decide again — so the wall-clock a user feels is the per-token…
Observability & debugging
Obstacle
You can't see why an agent did what it did
When an agent does the wrong thing, the run that produced it is a long, non-deterministic chain of model calls, tool results, and intermediate decisions — and most of that is…
Security & safety
Obstacle
Untrusted input and tools can hijack an agent
An agent treats whatever it reads — a web page, a tool result, a file, another agent's message — as instructions it might follow. Prompt injection turns that into an attack:…
Drift & maintenance
Obstacle
Agent behavior drifts as the model, SDK, and runtime churn under it
An agent is built on a substrate you don't control and that moves faster than your app: the underlying model gets upgraded or deprecated, the agent SDK and orchestration…
Solutions in this map
- Agent benchmarks: fixed tasks that exercise real tool use
- Context compaction: summarize, compress, and curate the working set
- Cost controls: budgets, metering, and per-task attribution
- External knowledge base: vector and graph retrieval
- LLM-as-judge: model-graded evaluation of traces and outputs
- Model Context Protocol: a standard interface for agent tools
- Orchestration patterns: topologies, handoffs, and harnesses
- Sandboxing, scoped credentials, and guardrails
- Speculative decoding: draft cheaply, verify in parallel
- Tracing and trace analysis for agent runs
- Version pinning, compatibility ranges, and staged upgrades