Context engineering

Context engineering concerns the selection, organization, and presentation of material supplied to a model at inference time. That material can include the user's question, governing instructions, retrieved passages, structured facts, and prior interaction. The term names a design problem rather than a single algorithm. A larger context is not automatically better: relevance, authority, timing, and the model's ability to use the supplied material all affect the result.

A sound practice states the task and the authority of each supplied item, keeps source boundaries visible, and tests whether the model uses the right evidence. Selection may involve retrieval, ranking, truncation, and formatting, with checks for omitted or conflicting material. Sensitive content and access permissions must be considered before inclusion. Stable identifiers and source locators make a generated statement easier to trace back to an underlying record.

For an organization defending each value, context engineering determines what evidence a model can consider, but it does not certify the evidence or the answer. Grounding and sentence-level citations provide observable links; faithfulness evaluation tests whether those links support the generated claims. Retrieval-augmented generation is one way to supply context, while the broader problem also includes task instructions, temporal scope, and the treatment of unresolved conflict.

Several failures have been documented. Long contexts are used unevenly: the study Lost in the Middle found that models answered best when the relevant passage was at the start or the end of the context and worse when it sat in the middle. Retrieval can miss the right passage, or return it without enough surrounding text to be understood; the paper Seven Failure Points When Engineering a Retrieval Augmented Generation System lists such points from practice, among them missing content, answers present in the retrieved material but not extracted, and answers in the wrong format or of the wrong specificity. Compressing context to fit a budget can discard the qualifier that mattered, as the article on context compression and evidence loss tests.

Design for inspection. Log what was in the context for each answer, with identifiers and versions of the sources. This is the context-side form of provenance, and without it, an answer that turns out wrong cannot be traced to a retrieval fault, a source fault or a generation fault. The article on RAG fault isolation describes how to separate those causes once the logs exist.

Pick ten answers and reconstruct their contexts from the log. For each, ask whether the evidence needed was present, whether anything misleading was present, and whether anything unauthorised was present. Ask what happens when nothing relevant is retrieved. Ask how long contexts are ordered and trimmed. Run the same question with the key passage moved from first to middle and see whether the answer changes, as in the article on long-context robustness to evidence position.