Context Engineering for AI Agents: A Practical Guide

Context engineering, stated plainly

Context engineering is the practice of deciding which tokens a language model sees at each step of a task, where those tokens come from, and in what form. For an agent that calls tools, reads documents and keeps notes across many turns, the context is assembled anew at every step from instructions, tool definitions, retrieved facts, memory and history. The engineering problem is to fit the most useful evidence into a finite and unevenly used window.

This guide treats context as a supply chain. Every token has an origin, and for work that must be defended later the question is not only "is this relevant?" but "can this be traced to a source, and should it be here at all?"

Where the term comes from, and who defines it how

Several definitions circulate; they overlap but are not identical.

• Anthropic (29 September 2025). "Context refers to the set of tokens included when sampling from a large-language model (LLM)." Context engineering "refers to the set of strategies for curating and maintaining the optimal set of tokens (information) during LLM inference, including all the other information that may land there outside of the prompts." It calls context engineering "the natural progression of prompt engineering" ^[1].
• Andrej Karpathy, as quoted by LangChain (2 July 2025). Context engineering is the "delicate art and science of filling the context window with just the right information for the next step" ^[2].
• Mei et al. (July 2025). The survey describes context engineering as "a formal discipline that transcends simple prompt design to encompass the systematic optimization of information payloads for LLMs". It formalizes the context as an assembly C = A(c1, ..., cn) of components and frames the discipline as an optimization of the context-generating functions under the hard constraint |C| ≤ Lmax ^[3].

None of these sources claims to have coined the term. The site's glossary entry for context engineering gives a shorter, vendor-neutral definition.

Prompt engineering versus context engineering

Anthropic contrasts the "discrete task of writing a prompt" with context engineering, which "is iterative and the curation phase happens each time we decide what to pass to the model" ^[1]. Mei et al. tabulate the same contrast in formal terms ^[3].

The last row is this article's addition. The system prompt remains one component among several.

The components of an agent's context

Anthropic's earlier post on agents distinguishes workflows, "systems where LLMs and tools are orchestrated through predefined code paths", from agents, "systems where LLMs dynamically direct their own processes and tool usage". Its basic building block is "an LLM enhanced with augmentations such as retrieval, tools, and memory" ^[4]. By September 2025 Anthropic had settled on a shorter definition of agents: "LLMs autonomously using tools in a loop" ^[1]. The loop is what makes context a supply-chain problem: each tool call adds material the next step may not need.

ReAct, which interleaves reasoning traces with actions and feeds each observation back into the prompt, is an early research formulation of such a loop ^[5]. Mei et al. name six context components ^[3]. The table maps them to their usual origin and the provenance question each raises.

Instructions and tools are authored artefacts, versionable like code. Retrieved knowledge and memory carry most claims about the world and are least likely to arrive with their origin attached.

Why curation is necessary: finite attention, position and context rot

Three lines of evidence explain why a larger window does not remove the need to choose.

Attention is a budget. Anthropic argues that context "must be treated as a finite resource with diminishing marginal returns", that LLMs have an "attention budget", and that transformer attention creates "n² pairwise relationships for n tokens" which get "stretched thin" as length grows ^[1].

Position matters. Liu et al. varied where the answer-bearing document sat among retrieved documents and found that performance "is often highest when relevant information occurs at the beginning or end of the input context, and significantly degrades when models must access relevant information in the middle of long contexts, even for explicitly long-context models." In one multi-document setting, GPT-3.5-Turbo's accuracy with the relevant document in the middle fell below its closed-book accuracy of 56.1%, that is, below answering with no documents at all. In an open-domain case study, using 50 retrieved documents instead of 20 improved performance by only about 1.5% for GPT-3.5-Turbo and about 1% for Claude-1.3 ^[6]. These were 2023 models; the site's evidence position robustness test describes how to measure the effect on the model you actually deploy.

Length alone degrades performance. Chroma's technical report evaluated 18 models while holding task difficulty constant and varying only input length, and found that "model performance consistently degrades with increasing input length". On a LongMemEval subset, models did markedly better on focused prompts of about 300 tokens than on the full prompts of about 113,000 tokens that contained the same answer plus irrelevant material. Topically related distractors had non-uniform effects that grew with length ^[7]. Anthropic links the term "context rot" to that report.

Strategies: select, compress, isolate, persist

LangChain groups agent context strategies into four buckets: write ("saving it outside the context window"), select ("pulling it into the context window"), compress ("retaining only the tokens required to perform a task") and isolate ("splitting it up") ^[2]. This guide calls the first "persist".

MemGPT makes the hierarchy explicit: "main context" (the prompt tokens: system instructions, a "working context" and a FIFO message queue) and "external context" in recall and archival storage. When prompt tokens exceed a warning threshold (the paper's example is 70% of the window), the system inserts a "memory pressure" warning so the model can save important information; at the flush threshold (for example 100%), it evicts messages (for example 50% of the window) and writes "a new recursive summary using the existing recursive summary and evicted messages" ^[8]. The summary that stays in context is a model-generated derivative of a derivative.

Anthropic's caution about compaction applies to every compressing step: "overly aggressive compaction can result in the loss of subtle but critical context whose importance only becomes apparent later" ^[1]. For work that must be defended, the "subtle but critical context" is often the date, the qualifier or the source locator. The site's compression evidence-loss acceptance test scores exactly that.

Facts need provenance before they enter context

An agent's answer can be no better sourced than its context. ReAct gives an early measurement of why grounding matters. On a sample of HotpotQA trajectories, hallucination accounted for 56% of chain-of-thought failures and 0% of ReAct failures, because ReAct read from an external knowledge base. The same analysis found the cost: for ReAct, "non-informative search, which counts for 23% of the error cases, derails the model reasoning" ^[5]. Retrieval helps only when what it returns is relevant and sound.

The reverse also holds. LangChain cites Drew Breunig's list of long-context failure modes, which begins with context poisoning, "when a hallucination makes it into the context", and includes context clash, "when parts of the context disagree" ^[2]. A memory note written from an earlier, unsupported answer looks identical in the window to one copied from a registry record.

A defensible context therefore applies an admission rule before the relevance rule. Each item that asserts something about the world should carry, at minimum: the source identifier, the source version or retrieval time, whether the text is verbatim or derived, and who or what produced it. Items without those fields may be true, but they can only prompt a lookup; they should not be cited.

Worked example: Halden Therapeutics AG

Halden Therapeutics AG is a fictional Basel pharmaceutical company. The product code, registry, documents and token counts below are invented for illustration. The token counts are illustrative, not measurements.

A compliance analyst asks an internal research agent: "What is the current development phase of HLD-204, and when is the next data readout expected?" Before the answering step, the orchestrator holds about 44,650 tokens of candidate material.

Context budget worksheet

Three decisions carry the method.

Row 7 is the trap. The memory note is short, relevant and confident, so a relevance-only rule would keep it. It has no source; admitted, it would let the agent state a readout date nobody published, the context-poisoning pattern above. The worksheet converts it into a lookup, which finds nothing, so row 11 records the date as unknown with the sources searched.

Row 5 compresses without losing evidence. Instead of summarizing the registry text, the orchestrator extracts the exact span and keeps its locator, so the compressed form is a quotation, not a paraphrase:

<evidence id="E1" kind="verbatim"
          source="https://registry.example.org/HLD-204" version="9"
          observed="2026-03-11T08:14:03Z">
Phase 2 study of HLD-204 in KRAS-mutant tumours
</evidence>

Rows 3 and 4 use position deliberately. Given Liu et al.'s results, the decisive evidence and the question sit at the end, not between conversation summaries.

The answer states "Phase 2, per registry entry version 9 observed 11 March 2026", cites E1, and reports that no published source for the readout date was found, rather than filling it from memory.

Failure modes and how to detect them

Each detector depends on logging what entered the context at every step, with origins; otherwise a failure is visible but cannot be attributed.

Checklist for a defensible context

1. Version instructions and tool definitions; record them per run.
2. Expose only the tools the step needs.
3. Admit factual items only with source, version or observation time, and a verbatim-or-derived flag.
4. Treat model-written memory as a lead, not as evidence, until it resolves to a source.
5. Compress evidence by extracting spans with locators; summarize only conversation.
6. Clear raw tool results once they have been used.
7. Place decisive evidence where the deployed model uses it best, and test that.
8. Detect conflicts before generation and require disclosure.
9. Record unanswerable fields as declared unknowns, with the sources searched.
10. Log the assembled context, with origins, at every step.

Anthropic's overall advice is to "find the smallest set of high-signal tokens that maximize the likelihood of your desired outcome" and, for agent design, to find "the simplest solution possible" ^[1][4]. Provenance adds one condition: high signal also means traceable.

This is the problem Reliable Context's published guarantees address: a cited, verbatim, dated source behind every value, and unknowns declared, never blank.