Skip to content
InovagioInovagio

Series · Knowledge layer deep dives

Knowledge layer: questions and answers

Straight answers to the questions buyers and architects ask about the knowledge layer for AI agents, from ontologies, knowledge graphs and semantic layers to context engineering, GraphRAG, semantic caching, small models and who owns it all.

· Published · Updated · 12 min read

On this page

A knowledge layer for AI agents is everything an agent should read before it acts on your work: your words for things and the rules between them; your records and what happened on past cases. The terms and rules are approved by the people who own them. Below are the questions we hear most, from buyers asking what it's for and architects asking how it's built. The answers are short; each links to the piece that goes deeper, and every piece is collected under knowledge.

Knowledge layer for AI agents: the basics

#

What is a knowledge layer?

#

It's the part of an AI system that holds what your business knows, as opposed to what a model learned from the internet. That means agreed terms, the categories they sit in, the rules that must hold, governed definitions of your numbers and a memory of what happened on past work. The model does the reasoning. The knowledge layer is what it reasons about.

The parts of a knowledge layer for AI agents as stacked bands: vocabulary and taxonomy, ontology and rules, the knowledge graph, governed metrics and memory of past work
What an agent reads before it acts. The words, rules and numbers each have an owner who approves them.

Inovagio splits the job in two: Synesis builds and governs the knowledge, and MNEMOS is the data engine that stores it next to your records. Every Virtual Employee reads from it. Our architecture page shows where it sits among the other layers.

What is a context layer, and is it the same thing?

#

Close enough that the difference is mostly vocabulary. Analysts converged on the name in 2026, and Forrester's definition is the most precise: a context layer combines the business semantics and governance of a semantic layer with the ontology modeling of a knowledge graph, and keeps adding runtime context such as events, decisions, actions and outcomes (Forrester, August 20, 2026).

Forrester's description of the context layer as the next evolution of semantic layers and knowledge graphs
Source: Forrester, The Next Evolution Of AI Will Rely On Context Layers, August 20, 2026.

The dated analyst sources, Gartner's forecast and what goes into one are in what a context layer is, and why AI agents need one. One caution when you read vendor recaps: Gartner's May 2026 release talks about semantics and context and never uses the words ontology or knowledge graph. Those are how you build the meaning, not what Gartner said.

What's the difference between a taxonomy, an ontology and a knowledge graph?

#

A taxonomy arranges your terms into parents and children. A thesaurus adds related terms and scope notes, and links one department's terms to another's. An ontology names the relationships between the things those terms name and states their rules, in a form software can reason with and check. A knowledge graph is your real records linked under that ontology. In our method they're stages, built in order, because each one depends on the agreements made in the one before it: vocabulary, taxonomy, thesaurus, ontology, knowledge graph. Taxonomy, ontology, knowledge graph explains each with a commercial lending example, and the ontology guide gives the W3C standards and code for each stage.

How is an ontology different from a data catalog or a business glossary?

#

A data catalog is an inventory of your datasets: where they live and who owns them. A business glossary defines terms in prose for people. An ontology defines the things those terms name formally, with the relationships between them, and its SHACL shapes add the constraints software can check, such as "an invoice is billed against exactly one purchase order". The glossary tells a person what "purchase order" means. The shape lets an agent notice that an invoice with two purchase orders is broken. (OWL alone wouldn't: told the link is single-valued, a reasoner concludes the two purchase orders are the same one.) Plenty of organizations have a catalog and a glossary and no ontology at all, and that's a fine place to begin, because a good glossary is the fastest way to start the vocabulary.

Building it

#

Do we need a knowledge graph?

#

Not on day one, in most cases. Forrester calls the semantic layer the right starting point and says most organizations aren't yet ready for a knowledge graph (Forrester, June 2, 2026). We agree. Build one when your questions cross systems: which loans in covenant breach are secured by a building with a stale appraisal, which suppliers share a bank account with an employee. The evidence for doing it then is strong. In one insurance benchmark, a model answered 16% of business questions correctly from the raw tables and 54% from a knowledge graph of the same data (Sequeda, Allemang and Jacob, November 13, 2023).

Who maintains it?

#

The people who own the words, and you decide who they are. Synesis lets you set who approves each stage and each kind of rule. Many organizations have a department head approve their own area's terms and taxonomies, and a governance body or compliance approve the ontology and the regulated rules; a rule can require more than one approver. Every change arrives as a change set (proposed, in review, approved, published), and by default, agents read only knowledge that has passed its gates. How a rule is approved shows the same discipline for one business rule, and why the knowledge build stalls covers what goes wrong when it's left to a project team.

How long does a first vocabulary take?

#

It depends on scope more than anything else, so keep the first scope to one queue. The terms come from interviews with the people who do the work and from the records they handle; why we start by listening describes the interviews. Agreeing a vocabulary for one team is quick. Agreeing it across departments that use the same word differently takes longer, and settling that disagreement, with a separate entry for each meaning and a mapping where two departments mean the same thing, is the thesaurus stage's job.

Which standards should it use?

#

Open ones, so you can export it. W3C SKOS (2009) for the first three stages, built to the practice in ANSI/NISO Z39.19 and ISO 25964. OWL 2 (2009, second edition 2012) to describe classes and relationships a reasoner can infer from, SHACL (2017) to validate records against constraints, SPARQL 1.1 (2013) to query, R2RML (2012) to map your existing tables and PROV-O (2013) to record where each fact came from. For governed metrics, the open semantic-layer format entered the Apache Incubator in July 2026 as Apache Ossie, and it's still incubating. The ontology guide explains when to use each; it also explains why you need both OWL and SHACL, which most teams learn the hard way.

A table of open standards and what each does: SKOS for vocabulary, taxonomy and thesaurus; OWL 2 for classes a reasoner can infer from; SHACL to validate records; SPARQL 1.1 to query; R2RML to map existing tables; PROV-O to record where each fact came from
Synesis imports and exports RDF, OWL, SKOS and SHACL, so the ontology and its rules can leave with you.

What is a semantic layer, and do agents need one?

#

A semantic layer defines your numbers once (metrics, dimensions and the joins between tables) so an agent asks for "loss ratio" instead of writing its own SQL. Agents writing SQL fail with plausible wrong numbers; a semantic layer fails with an error when a question is out of scope. In a 2026 paired benchmark, a 4 KB document of business definitions added 17 to 23 points of accuracy for each of three frontier models (Rumiantsau and Fokeev, April 28, 2026). The worked example and the rest of the evidence are in semantic layer for AI agents.

Using it with agents

#

What is context engineering?

#

It's choosing what goes into the model's context at each step of a task. The most-cited practitioner article on it describes context as a finite resource with diminishing returns, uses the name "context rot" for the way recall degrades as the window fills, and recommends the smallest set of high-signal tokens that does the job, using just-in-time retrieval, compaction, structured notes kept outside the window, and sub-agents with clean contexts (Rajasekaran et al., "Effective context engineering for AI agents", September 29, 2025).

Timeline of context moving from a technique to infrastructure: context engineering in September 2025, a context layer as core infrastructure in Gartner's May 2026 release, and AI context platforms among Gartner's 2027 trends in October 2026
Sources: Rajasekaran et al., September 29, 2025; Gartner, May 11 and October 20, 2026.

The knowledge layer is what makes that possible: you can only retrieve just in time if there's something approved to retrieve. Context engineering for AI agents walks through what one step's context actually contains.

#

Vector retrieval finds passages similar to the question; GraphRAG retrieves through a graph, either one a model extracts from your documents or one curated against your ontology. A systematic comparison first published in 2025 found each approach stronger on different kinds of question, and running both and merging their results beat either one alone on a multi-hop benchmark (Han et al., "RAG vs. GraphRAG: A Systematic Evaluation", 2025, revised 2026).

For regulated work, an extracted graph is cheaper to build and noisier than a curated one, and running both kinds of retrieval means paying for both on every query, so route single-fact questions to text search alone. GraphRAG vs RAG in our knowledge graph guide covers the research and when the graph earns its cost.

How does an agent remember what happened?

#

Through memory kept outside the model, in layers. MNEMOS keeps five: raw records, working memory for the task in hand, episodic memory of past tasks, semantic memory of facts and shared memory for the Virtual Employees working one queue. None of these is where the rules live: what an agent learns from its own history is memory, and it reaches the approved knowledge only as a proposal the right person approves. Memory is also an attack surface. OWASP's Top 10 for Agentic Applications lists memory and context poisoning among the ten risks (OWASP GenAI Security Project, December 9, 2025), which is why every write to shared memory should be attributed and every read trimmed to what the user may see.

What is a semantic cache, and should we use one?

#

A semantic cache stores model answers keyed by the meaning of the question, so a paraphrase can be answered without calling the model. It cuts model calls. It can also serve the wrong answer: "what does it cost to cancel in the first year?" and "what does it cost to cancel after the first year?" sit very close together in embedding space and have different answers. For regulated work, cache only reference answers that don't depend on the person asking, scope every key by tenant, role and policy version, expire entries when the source changes, and log each hit. In Inovagio, MNEMOS caches the knowledge layer's own lookups rather than model answers, such as the slice of the ontology an extraction prompt needs or a metric's latest result. Each entry's key carries the reader's tenant and clearance as well as the knowledge version, and the entry is evicted the moment the approved knowledge it came from changes, never on a timer. The Inference Server has a different kind of cache: a radix prefix cache that survives across turns, so work already computed for the same prompt prefix isn't repeated. That's exact-match reuse of computation, not a cache of answers.

Rules and models

#

How is a business rule different from an AI guardrail?

#
A guardrail beside a business rule: the guardrail filters what a model may say with probabilistic or pattern checks; the rule checks what the work must satisfy, with the same result for the same input and its ID on the record
Regulated work needs both.

A guardrail filters what goes into a model and what comes out: prompt injection, leaked personal data, unsafe content. It's usually probabilistic or pattern-based, and it sits around the model. A business rule states what the work must satisfy, such as "a supplier payment over the clerk's limit goes to a manager", and it's checked deterministically: same input, same result, with the rule's ID and version on the record.

Regulated work needs both. Business rules or AI guardrails sets out the difference with an example, and Phylax is where approved rules and policy checks are enforced in Inovagio.

Do we need a large model, or will a small one do?

#

For most steps of most business tasks, a small model will do. An NVIDIA Research position paper cites estimates that serving a 7-billion-parameter model is 10 to 30 times cheaper than serving a 70 to 175 billion one. Its authors also estimate that 40% to 70% of the model calls in three open agent frameworks could go to small models. Neither figure is a measurement (Belcak et al., "Small Language Models are the Future of Agentic AI", 2025, revised 2026). Gartner predicts that by 2027 organizations will use small, task-specific models at least three times more than general-purpose large models, by usage volume (Gartner, April 9, 2025).

Two figures from an NVIDIA Research position paper: serving a small model estimated at 10 to 30 times cheaper, and 40% to 70% of agent model calls estimated replaceable by small models
Source: Belcak et al., arXiv 2506.02153, 2025, revised 2026. Both figures are estimates, not measurements.

The limit is long, multi-step planning. Small and mid-size open models handled short, structured tool use well in the AgentFloor benchmark, while the frontier model kept its edge over the strongest open model on long-horizon planning, though every model failed most of those tasks (Karmakar and Chatterjee, May 1, 2026). Small language models for business gives the plain-English case, SLMs for enterprise AI agents covers routing and fine-tuning, and Origin is where small models you own take the routine work, with frontier models only when you choose.

Who owns the knowledge layer if we change vendors?

#

You should. The knowledge layer is the asset that took your people's time to build, and it outlasts any model. Insist on open formats and an export: Synesis imports and exports RDF, OWL, SKOS and SHACL, so the ontology and its rules can leave with you. The case for owning the knowledge rather than renting the people who hold it is in engineers you rent, or AI you own.

Can we turn the agents off without losing the knowledge?

#

Yes, and the two should be separate. The knowledge lives in the knowledge layer and the agents only read it, so stopping them leaves the vocabulary and the rules exactly where they were, approved and versioned as before. Archon has a kill switch for one Virtual Employee, a department or everything at once, and keeps every action on the record. For what examiners now ask about stopping agents, read can you turn your AI agents off?.

Further reading

#
  • Gartner Says Lack of Semantics Causes Inaccurate AI Agents and Wasted Spending, Gartner, May 11, 2026: gartner.com
  • Gartner Identifies the Top Strategic Technology Trends for 2027, Gartner, October 20, 2026: gartner.com
  • The Next Evolution Of AI Will Rely On Context Layers, Forrester, August 20, 2026: forrester.com
  • Build Meaning Before Machines, Forrester, June 2, 2026: forrester.com
  • Effective context engineering for AI agents, Rajasekaran, Dixon, Ryan and Hadfield, engineering article, September 29, 2025
  • From Local to Global: A Graph RAG Approach to Query-Focused Summarization, Edge et al., 2024, revised 2025: arxiv.org
  • RAG vs. GraphRAG: A Systematic Evaluation, Han et al., February 2025, revised November 7, 2026: arxiv.org
  • Small Language Models are the Future of Agentic AI, Belcak et al., NVIDIA Research, 2025, revised 2026: arxiv.org
  • AgentFloor, Karmakar and Chatterjee, May 1, 2026: arxiv.org
  • OWASP Top 10 for Agentic Applications for 2026, OWASP GenAI Security Project, December 9, 2025: owasp.org

Share

Questions

What people ask.

What is a knowledge layer in AI?

A knowledge layer is the part of an AI system that holds what your business knows: agreed terms and categories, the rules between things, governed definitions of your numbers, and a memory of past work. The terms and rules are approved by the people who own them. AI agents read it before they act, so they work from your definitions instead of guessing.

Is a context layer the same as a knowledge layer?

Mostly, yes. Gartner and Forrester use context layer for much the same idea: a governed layer of business meaning, knowledge and runtime events that AI agents read at each step. Forrester describes it as the next evolution of semantic layers and knowledge graphs.

Do we need a knowledge graph before we use AI agents?

Usually not. Most teams start with an agreed vocabulary for one queue and governed definitions of their key numbers. A knowledge graph earns its cost when the questions you need answered cross several systems.

Who should maintain the knowledge layer?

The people who own the words, with the right person approving each change: a department head for their own area's terms, perhaps, and a governance body or compliance for the ontology and regulated rules. The most sensitive rules can require more than one approver. Every change should be versioned and approved before an agent can read it.

How is a business rule different from an AI guardrail?

A guardrail filters what goes into and comes out of a model and is usually probabilistic or pattern-based. A business rule states what the work must satisfy and is checked deterministically, with the same result for the same input and the rule's ID on the record. Regulated work needs both.

Can a small language model do the work of a large one?

For most short, structured steps of business tasks, yes, on far smaller hardware. Frontier models keep an edge on long, multi-step planning, where no model is reliable yet, so a good design routes routine steps to small models and sends the rest to a larger model when you choose.

See it on your work

See how one of your tasks runs, start to finish.

A 30-minute walkthrough of one of your workflows, the approval step and the audit record, with the people who build the platform.

We use these details to answer your request. See our privacy policy.

What happens next

  1. 1We reply within one business day to find a time.
  2. 2A 30-minute walkthrough of one of your tasks, start to finish.
  3. 3You leave with a short write-up of what we covered.