LLM Digest
Subscribe

Agent Engineering Wiki

Agent engineering · knowledge map

Agents give fluent, confident-looking output even when it's wrong

🧱 Obstacle·reliability·stub·2 sources·updated 2026-07-07

An agent can hallucinate a fact, skip a step, or misuse a tool and still return a fluent, confident-looking answer — nothing about the output itself signals that it's wrong. Deciding where to trust the model's own reasoning versus routing to a deterministic tool, and getting an agent to actually prove its work rather than just claim success, is a distinct engineering problem from measuring that work after the fact (see agent evaluation).

Evidence · 2 sources