Reliable prompts are interfaces, not prose. If an agent step matters, write the prompt like a contract: define the role of the step, the available inputs, the required output shape, the decision rules, and the failure behavior you will test.
"},{"heading":"Short answer","html":"A good prompt reduces ambiguity in the model's continuation while preserving the information needed to solve the task. Instructions set the goal, examples show the local pattern, schemas constrain the output channel, and evals tell you whether the contract survives model, context, and data changes.
"},{"heading":"Builder model","html":"Treat every important prompt as a small API. The input is the context you pass in, the implementation is the model's learned distribution, and the output contract is what downstream code or humans consume. Reliability improves when the prompt makes the desired continuation easier than nearby wrong continuations, then verifies that behavior with representative cases.
"},{"heading":"Mechanism","html":"An autoregressive language model predicts the next token conditioned on the tokens before it. A prompt is therefore not just an instruction; it is the conditioning environment for all later tokens. Clear task framing, examples, delimiters, and output schemas change which continuations are likely.
\nFew-shot examples work because they place a pattern directly in context. Chain-of-thought examples can help on multi-step problems because they demonstrate an intermediate representation before the final answer. Self-consistency helps when the model can reach the same answer through multiple sampled paths. Long context is not automatically reliable context: relevant information can be harder to use when it is buried in the middle.
"},{"heading":"Math intuition","html":"Think of the model as assigning probability mass across possible next-token paths. A vague prompt spreads mass across many plausible completions: explanation, refusal, partial answer, wrong format, hidden assumption. A reliable prompt concentrates mass around the acceptable region.
\nExamples act like local coordinates: they show the model what kind of mapping you want. A schema narrows the output subspace. Delimiters reduce accidental mixing between instructions, retrieved text, and user data. Evals estimate whether the probability mass stays in the right region across the cases you actually care about.
"},{"heading":"Evidence","html":"Write the prompt contract before polishing wording. Specify the task, the input fields, what the model must ignore, the output schema, and what to do when evidence is missing. Put volatile or untrusted content behind clear delimiters. Keep decisive instructions close to the work they govern, especially when the context is long.
\nFor agent systems, pair the prompt with a small eval set. Include clean successes, missing-information cases, adversarial or irrelevant context, and format-stability checks. Change one prompt dimension at a time, then compare outputs against the contract. If downstream code parses the answer, schema adherence is part of correctness, not formatting polish.
"},{"heading":"Failure modes","html":"Use agent evaluation to test whether prompt changes helped, context compaction to keep the working set usable, and prompt injection when the prompt includes untrusted text or tool output.
"}],"evidence":[{"id":"brown-2020-language-models","kind":"theory-paper","tier":"theory/paper-backed","title":"Language Models are Few-Shot Learners","note":"Shows that task behavior can be specified through instructions and examples in context, without gradient updates.","url":"https://arxiv.org/abs/2005.14165"},{"id":"wei-2022-chain-of-thought","kind":"benchmark-result","tier":"benchmark/result-backed","title":"Chain-of-Thought Prompting Elicits Reasoning in Large Language Models","note":"Reports that exemplars with intermediate reasoning improve performance on arithmetic, commonsense, and symbolic reasoning tasks.","url":"https://arxiv.org/abs/2201.11903"},{"id":"wang-2022-self-consistency","kind":"benchmark-result","tier":"benchmark/result-backed","title":"Self-Consistency Improves Chain of Thought Reasoning in Language Models","note":"Shows that sampling multiple reasoning paths and selecting a consistent answer can improve reasoning benchmark accuracy.","url":"https://arxiv.org/abs/2203.11171"},{"id":"liu-2023-lost-in-the-middle","kind":"benchmark-result","tier":"benchmark/result-backed","title":"Lost in the Middle: How Language Models Use Long Contexts","note":"Finds that long-context models can perform worse when relevant information appears in the middle of the context.","url":"https://arxiv.org/abs/2307.03172"},{"id":"prompt-reliability-editorial-synthesis","kind":"editorial-inference","tier":"editorial inference","title":"LLM Digest synthesis","note":"For agent builders, prompt reliability should be treated as an interface and evaluation problem, not a copywriting problem."}],"related_topics":[{"slug":"agent-evaluation","title":"Measuring whether an agent actually worked is hard"},{"slug":"context-compaction","title":"Context compaction: summarize, compress, and curate the working set"},{"slug":"prompt-injection","title":"Untrusted input and tools can hijack an agent"}],"related_playbook_cards":[],"related_storylines":[],"covers_evidence":["brown-2020-language-models","wei-2022-chain-of-thought","wang-2022-self-consistency","liu-2023-lost-in-the-middle","prompt-reliability-editorial-synthesis"]}}}