Skip to content
InovagioInovagio

Series · Knowledge layer deep dives

Context engineering for enterprise AI agents

Context engineering decides what an AI agent sees at each step of a task. The research on why long contexts fail, the techniques that keep context small, how agent memory fits in, and the controls that stop context from becoming an attack path.

· Published · Updated · 18 min read

On this page

An agent that fails on step nine of a twelve-step task rarely fails because the model is weak. Usually step nine was handed the wrong material: a stale rule, half of a contract, forty lines of tool output nobody needed. Context engineering for AI agents is the discipline of deciding what an agent sees at each step of a task: which instructions, rules, records, past work and tools go into the model's input, how they're laid out, and what stays out.

Prompt engineering was about writing one good instruction. An agent doesn't run on one instruction. Software rebuilds its input before every model call, dozens of times per task. That rebuild is where most of the system's accuracy is decided, and most of its cost and security exposure too.

This guide is for architects and developers. We start with the research on why long inputs degrade, then build one step's context by hand, budget included. The sections after that cover memory and retrieval, and the ways context becomes an attack path. If you want the business view first, read what a context layer is, and why agents need one.

Where the term came from

#

"Context engineering" caught on among practitioners in mid-2025, as teams moved from chat to agents that run for minutes or hours. In October 2025 Gartner called it the new prompt engineering, and defined it as designing and structuring the data, workflows and environment around an AI system so it understands intent and produces results aligned with the enterprise. Its advice was organizational as much as technical: appoint a context engineering lead or team, tie it to AI governance, and monitor context continuously for drift (Gartner, October 6, 2025). The article carries no forecasts, which is worth knowing if a vendor quotes a Gartner number about context engineering at you.

The most useful engineering account came a week earlier, in Effective context engineering for AI agents by Prithvi Rajasekaran and three colleagues (Ethan Dixon, Carly Ryan, Jeremy Hadfield), an engineering article published on September 29, 2025. They described context engineering as the work of curating and maintaining the best set of tokens for each model call. Tokens are the word pieces a model reads and is billed for. Most of what follows borrows that article's vocabulary. Where it rests on outside research, we link the research.

Why more context makes answers worse

#

A model with a million-token window can't just read everything. There are two documented reasons.

The first is architectural. In a transformer, each token can attend to every token before it, so n tokens create on the order of n² pairwise relationships. Rajasekaran and colleagues call the result an attention budget: each token added draws it down, and the model's grip on any one relationship gets weaker. The measurable effect has a name, context rot: a model recalls information less accurately as the token count grows. Independent tests bear it out. NoLiMa, a benchmark presented at ICML 2025, keeps the literal word overlap between a question and the passage that answers it to a minimum, so a model can't find the answer by matching words; it has to make the connection. By 32,000 tokens, 11 of the 13 models tested, all of them sold as handling at least 128,000, had fallen below half of their short-input score (Modarressi et al., NoLiMa, arXiv, February 2025). The advertised window and the usable window are different numbers.

Two numbers from the NoLiMa benchmark: 11 of 13 models fell below half their short-input score by 32,000 tokens, though each claimed a window of 128,000 tokens or more
Context rot, measured. Source: Modarressi et al., NoLiMa, ICML 2025.

The second is positional. Nelson Liu and colleagues at Stanford and elsewhere gave models a question and a set of documents, one of which held the answer, and moved that document around. Accuracy was highest when the answer sat at the beginning or end of the input and dropped when it sat in the middle, even for models built for long contexts (Liu et al., Lost in the Middle, Transactions of the Association for Computational Linguistics, 2024; arXiv July 2023). The curve is U-shaped. Put the contract clause that matters in the middle of 40 pages and the model may read straight past it.

A quotation on the guiding principle of context engineering for AI agents: find the smallest possible set of high-signal tokens
The guiding principle of the article this guide builds on. Source: Rajasekaran, Dixon, Ryan and Hadfield, September 29, 2025.

Together these give the field its working rule. The same article puts it as finding the "smallest possible set of high-signal tokens" that makes the desired result most likely. Everything else in this guide is a way to follow that rule without losing information the agent needs later.

There's a cost argument too. Every token in the input is paid for on every call. Gartner's 2027 trends list names "excessive token consumption" among the risks of running agents without a dedicated context layer (Gartner, October 20, 2026). An agent that re-reads a 60-page policy on each of twelve steps pays for 720 pages. Prompt caching reduces what a repeated prefix is billed; it does nothing for the attention that prefix competes for.

What one step's context should contain

#

Abstract advice gets vague fast, so here is one step from a real kind of task.

Calloway Building Supply runs an accounts payable queue (Calloway Building Supply · Example company). A freight carrier's invoice arrives with a fuel surcharge that isn't on the purchase order. The agent is on step four of the task: decide whether the surcharge is within the carrier's contract rate, and if not, route it to the controller.

A naive build stuffs in the whole contract, the vendor's full payment history, the AP policy manual and every tool the platform offers. A deliberate build hands over this:

yaml
step: ap-invoice/4-check-surcharge
goal: "Is the fuel surcharge on invoice INV-88214 within the contract rate?"
acting_for: { tenant: calloway, role: ap-specialist, queue: accounts-payable }
rules:                       # approved rules that apply to this step, by id and version
  - { id: AP-FREIGHT-07, version: 3, approver: controller }
  - { id: AP-ROUTE-02, version: 1, approver: ap-lead }
records:                     # fetched by id; only the fields the rules read
  - { invoice: INV-88214, fields: [carrier, lines, surcharge_amount, surcharge_basis] }
  - { purchase_order: PO-5512, fields: [lines, agreed_freight] }
  - { contract: CARRIER-2291, clause: "4.3", fields: [fuel_surcharge_table, effective_dates] }
past_work:                   # episodic memory: two similar cases, summarized
  - { date: 2026-09-18, case: "Same carrier, surcharge within rate", outcome: paid }
  - { date: 2026-10-02, case: "Same carrier, rate table expired", outcome: routed_to_controller }
tools: [lookup_fuel_index, route_to_person]
notes: "Steps 1-3 matched vendor, PO and receipt; no exceptions."
budget: { input_tokens_max: 6000 }
One agent step's context shown as a screen with five numbered parts: the goal, who it acts for, the approved rules, the record fields, and past work with two tools
Everything the step uses, and nothing it doesn't. Calloway Building Supply · Example company.

Four choices in that manifest do most of the work.

  • Rules go in by id and version. The agent gets the two approved rules that apply, not the policy manual. When an auditor later asks why the surcharge was held, the record names AP-FREIGHT-07 version 3 and its approver. Our guide on how a rule is approved shows what that rule looks like.
  • Records go in as fields. The agent reads clause 4.3, and the rest of the contract stays out. That keeps the deciding text near the start of the input, where Liu's results show models use it well.
  • Past work is two cases, summarized: one paid, one routed, both for this carrier. A list of every invoice from the carrier would add tokens and no judgment.
  • Only two tools are exposed. Each tool definition costs tokens and widens what the agent can do. A step that only needs to look up a fuel index and route a case shouldn't see a payments tool.

The whole manifest renders to a few thousand tokens. Everything the naive build stuffs in would cost far more, and it would bury clause 4.3 somewhere in the middle.

Five techniques that keep context small

#

The techniques below come from the Rajasekaran article and from production practice. They're complementary; most serious agents use all five.

TechniqueWhat it doesWhat to watch
Just-in-time retrievalThe agent holds lightweight identifiers (file paths, stored queries, record ids) and loads content with a tool when a step needs itRetrieval latency, and tools that return far more than asked
CompactionNear the limit, the history is summarized and a fresh window starts from the summaryLosing a decision or an open issue in the summary
Notes outside the windowThe agent writes progress and decisions to durable memory and reads them back laterNotes that drift from the record, or get poisoned
Sub-agentsA focused sub-agent explores in its own clean window and returns a short summarySummaries that drop the evidence the lead agent needs
Instructions at the right levelThe system prompt states principles and examples, not a brittle list of if-then casesPrompts that grow a new paragraph after every failure
A table of five context engineering techniques, what each does and what to watch
Each technique trades something away. The right-hand column is where production systems fail.

A few specifics are worth carrying over from the article. In its compaction example, the summary keeps architectural decisions and unresolved bugs, along with implementation details, and throws away redundant tool output; the lightest form is just clearing old tool results from history. Sub-agents may spend tens of thousands of tokens exploring and still return a condensed summary, often 1,000 to 2,000 tokens long. The authors also describe a hybrid: load a small set of standing instructions up front and fetch everything else just in time, which they suggest suits slower-moving content such as legal or finance work.

That hybrid is close to how regulated work should run. The approved rules for a queue change slowly, so they can be loaded up front by id and version. The records change with every task, so they're fetched when the step needs them.

Our Agent Harness, the runtime every Inovagio face shares, plans the work, breaks it into tasks, keeps durable memory across sessions and compacts context as a session grows. Nexus, which runs agents across the network, adds a handoff protocol that summarizes context when one agent passes work to another, and session budgets for tokens, calls and cost.

AI agent memory is part of context

#

An agent that forgets everything between steps has to be told it all again, and that costs tokens. An agent that remembers everything carries noise. Memory design sits between the two. It helps to separate memory by what it holds and how long it lives.

Five layers of AI agent memory, from raw records at the base to memory shared by the Virtual Employees on one queue at the top
Each layer holds something different and lives for a different length of time.

MNEMOS, our data engine, keeps five layers of agent memory:

  • Raw memory is every observation, unchanged and never edited: the invoice as received, the email, each tool call and its output. It's the evidence, and it's rarely put into context whole.
  • Working memory is scratch state for the turn in progress, and it's cleared when the turn ends. Anything a later step needs, like the notes in the manifest above, is written down before then.
  • Episodic memory is what happened in past tasks and sessions and how each ended, searchable by time, by similarity and through the graph. This is where "same carrier, rate table expired, routed to controller" comes from.
  • Semantic memory holds facts and preferences distilled from many episodes by background reflection jobs. Each entry links back to the episodes it came from, and entries that stop being used fade to cold storage instead of crowding recall.
  • Shared memory holds procedures and knowledge, shared by the Virtual Employees on one queue with access controlled per entry, so the second one doesn't rediscover what the first one learned.

The layers matter for context because each one is retrieved differently. Working memory is read in full because it's small. Episodic memory is searched for similar past cases. Semantic memory is searched by meaning. Raw memory is cited when someone needs the evidence. And MNEMOS recalls against a token budget rather than a fixed top-k: it plans keyword search, vector search and reranking as a cascade, counts the tokens of each candidate and returns a set that fits.

None of those layers is where the rules live, and that's deliberate. What an agent learns from its own history is memory. What the business approved is knowledge, and it sits in the knowledge graph that Synesis, our knowledge layer, publishes only after the right person approves it. Virtual Employees read that graph; by default they see only published, fully approved content. A Virtual Employee can propose a new fact for it, but it can't publish one. Keeping the two apart is the main defense against an agent that learns a bad habit from its own history, and it's why the pillar guide on ontology for AI agents spends so long on stewards and change sets.

Retrieval decides what reaches the window

#

Most of an agent's context arrives through retrieval. So retrieval quality caps context quality.

Keyword search (usually BM25, the standard ranking formula in full-text engines) finds exact terms like a contract number. Vector search finds passages with similar meaning even when the words differ. Neither knows that a "fuel surcharge" and a "fuel adjustment factor" are the same thing in your business unless someone wrote that down. MNEMOS runs keyword and vector retrieval together and merges the two ranked lists with reciprocal rank fusion. That method, from a 2009 information retrieval paper, scores each result by the sum of 1/(k + rank) across the lists, with k set to 60 in the original experiments (Cormack, Clarke and Büttcher, SIGIR 2009). Fusion uses only ranks, so it doesn't matter that a BM25 score and a cosine similarity are on different scales. A cross-encoder reranker, served by the Inovagio inference server, then reads each surviving passage against the query and reorders the short list. Synesis runs concept-expanded search across SQL, full-text and vector stores, so a search can reach documents that use another approved name for the same thing.

For questions that cross several records (this invoice, under this contract, from a carrier whose rate table expired last quarter), graph-walking retrieval follows the connections in the knowledge graph instead of hoping a single passage holds the answer. When that extra machinery earns its cost is the subject of our guide to how AI agents use your knowledge graph.

MCP connects tools; it doesn't supply meaning

#

The Model Context Protocol (MCP) standardizes how an agent's host application connects to servers that offer tools, data (which MCP calls resources) and prompt templates, using JSON-RPC messages. It was released as an open standard in November 2024, and in December 2025 it became a founding project of the Agentic AI Foundation under the Linux Foundation, which cited more than 10,000 published MCP servers (Linux Foundation, December 9, 2025).

MCP is plumbing, and good plumbing. It says how a tool's description and results reach the model. Its authorization spec, built on OAuth 2.1, decides whether a client may call a remote server at all. It says nothing about whether status = R means reopened, which records this user may see, or whether the rule the tool applies is the approved version. An agent with 50 MCP servers attached and no context discipline has more ways to fill its window with noise, and each tool definition loaded into the window costs tokens on every call.

Two practical consequences follow. Expose tools per step, not per agent, as the manifest does. And route MCP calls through the same authorization and audit as everything else: in our platform, Connect provides governed connectors and MCP servers where every call is authorized and audited, and gets its secrets from the secrets store. Guard screens every tool call (MCP calls included) for prompt injection and personal data.

Context is an attack surface

#

Anything that can write into an agent's context can steer it. OWASP's Top 10 for Agentic Applications, published December 9, 2025, lists this as ASI06, memory and context poisoning, alongside goal hijack, tool misuse and identity and privilege abuse (OWASP GenAI Security Project, December 9, 2025). The danger is persistence. A poisoned memory keeps shaping behavior long after the conversation that planted it. In a May 2026 post, a co-lead of that OWASP entry described attacker-controlled content from a routine developer workflow reaching an agent's persistent memory and system prompt, and argued that memory deserves the same scrutiny as credentials (Habler, OWASP GenAI Security Project, May 13, 2026).

Permissions are the other half. If retrieval runs with broad access and the agent is trusted to ignore what it shouldn't see, one injected instruction can ask it to repeat that material. The fix is to make sure the forbidden material is never retrieved at all.

The order in which one step's context is assembled: identify who the agent acts for, scope to the tenant and trim to permitted records, retrieve, screen, then fit the budget
Permission checks come before retrieval results reach the model, never after.

Assemble context in a fixed order, because the order is the control. This is the order we recommend, with what our platform does at each point:

  1. Identify. Every step runs as a known person or service, and each Virtual Employee has an account of its own, one you can disable. The NIST National Cybersecurity Center of Excellence's February 2026 concept paper on agent identity and authorization frames the same requirements: identify the agent, authorize each action, audit it (NIST NCCoE, February 5, 2026).
  2. Scope to the tenant. In MNEMOS every record is bound to the signed-in tenant in code, and Synesis keeps its knowledge objects tenant- and area-scoped with fail-closed isolation.
  3. Trim. MNEMOS applies document-level security trimming to every result, with the security filter inside the search rather than on its output, so a document the identity may not see is never a candidate, never reaches the model and can't be quoted. An agent's access is capped at that of the person or service it acts for.
  4. Retrieve and rank. Retrieval of every kind (keyword, vector, graph) runs inside that scope.
  5. Screen. Treat retrieved text as data to read, never as instructions to follow, and keep it in a separate, labeled part of the input. Guard screens every prompt and every response for prompt injection and personal data, and every tool call as well.
  6. Fit the budget. The step's token budget is applied last, dropping the lowest-ranked material first and keeping approved rules whole.

Memory writes need the same care in reverse. In Synesis an agent may propose a fact for the knowledge graph, and nothing is used until the right person approves it. An agent run that has read untrusted text, such as a web page, a transcript or an unreviewed candidate, holds no permission to approve or publish anything. Every memory entry should carry where it came from, so a suspicious one can be traced and removed; in MNEMOS each distilled memory links back to the events it was drawn from. MNEMOS also records every write in a tamper-evident ledger, which means you can find what a poisoned entry touched.

Measure context, then tune it

#

You can't tell whether context is right by reading it once. Treat the context recipe for each step as something you test and version.

A four-step cycle for tuning context: run the step, check the result, find what was missing or noise, change the recipe with approval
The recipe for each step changes the way a rule does, with a test and an approval.

Run the step on a fixed set of past cases with known answers. Check each result against the approved rules and the records it used. For the failures, look at what the context held: was the deciding fact missing, or present but buried? Then change the recipe and run the set again.

Measure retrieval on its own as well, with queries whose relevant documents someone has judged. Recall at k tells you whether the deciding passage was retrieved at all; nDCG@10 (normalized discounted cumulative gain over the top 10 results) tells you whether it was ranked high enough for the budget to keep it. A wrong answer with the right passage in context is a reading problem. A wrong answer without it is a retrieval problem, and no prompt change will fix it.

Three numbers tell you most of what you need: accuracy on the case set; input tokens per step, because a recipe that grows every week is drifting; and cost per correct task, which folds both together. Our guide to measuring cost per correct task gives the formula. Every Inovagio model call is recorded with the model used and its token count and cost, so these come from the task records rather than estimates, and reading a task record shows where each one appears.

A context engineering checklist for your first agent

#
  1. List the steps of one task, and for each step write the question it answers.
  2. For each step, name what it needs: rules, record fields, past work, tools. Leave out everything else.
  3. Put approved rules in by id and version. Never paste a policy manual.
  4. Fetch records by id when the step needs them, and keep the deciding fields near the start of the input.
  5. Decide what each memory layer receives (working, episodic, semantic), and who approves semantic entries.
  6. Bind every retrieval to the tenant and trim to the caller's permissions before ranking.
  7. Treat every retrieved passage and tool result as data, and screen it.
  8. Set a token budget per step and record actual use.
  9. Test the recipe on known cases before you change it, and version it when you do.

If your agents don't yet share one meaning for each term in a single queue, none of this will hold, so start with the vocabulary; our knowledge layer questions and answers cover where most teams begin.

Further reading

#

Share

Questions

What people ask.

What is context engineering?

Context engineering is deciding what an AI agent sees at each step of a task: which instructions, approved rules, records, past work and tools go into the model's input, in what order and form, and what is left out. It replaced prompt engineering as the main design work once agents began running many steps, because software rebuilds the input before every model call.

How is context engineering different from prompt engineering?

Prompt engineering is writing one good instruction for one model call. Context engineering is designing the system that assembles the input for every call an agent makes, including retrieval, memory, tool definitions, permissions and a token budget. A well-written prompt is one part of it.

Why not just use a model with a very large context window?

Because models recall less as the input grows, which practitioners call context rot, and they tend to miss information placed in the middle of a long input. In one 2025 benchmark, most models that advertise 128,000-token windows fell below half their short-input score by 32,000 tokens. Every token also costs money on every call. A smaller input that holds the right facts near the start usually gives better answers, on fewer tokens, than a larger one that holds everything.

What is memory and context poisoning?

It is an attack where untrusted content gets into an agent's memory or retrieved context and steers what the agent does later, sometimes across many sessions. OWASP lists it as ASI06 in its Top 10 for Agentic Applications. Defenses include keeping approved knowledge apart from what agents learn, recording where each memory came from, screening retrieved text and filtering retrieval to the caller's permissions before the model sees anything.

Does the Model Context Protocol solve context engineering?

No. MCP standardizes how an agent's host application connects to servers that offer tools, data and prompt templates. It does not decide which tools a step should see, what business terms mean, which records a person may read or which rule version applies. Those decisions still need a context layer and an assembly process with authorization and audit.

How do you know whether an agent's context is right?

Test it. Run each step on a set of past cases with known answers, check the results against the approved rules, and look at what the context held for every failure. Track accuracy, input tokens per step and cost per correct task, and version the context recipe when it changes.

Developer edition

Build on Inovagio.

The MNEMOS data engine, the command line and the desktop app, with guides and API references.

This is for

We use these details to answer your request. See our privacy policy.

What happens next

  1. 1We reply to every request.
  2. 2We send the next steps for your team.
  3. 3A direct line to the engineers who build it.