By Jeremy Arendt
The language we use to describe data is shifting faster than products evolve: every quarter a vendor introduces a new approach, coins a new term, or adopts and reshapes someone else’s term to match their product. That’s not malicious—companies describe the landscape from their vantage point—but when six vendors give six meanings to the same word, practitioners spend meetings translating definitions before any design conversation can start.
A second issue compounds this: much of today’s vocabulary comes from academic disciplines that few practitioners have deep experience with. "Data warehouse" is intuitive; "ontology" is not. Tom Gruber defined ontology in 1993 as an explicit specification of a conceptualization—precise, but of limited use to a director choosing next quarter’s investments.
Below is a practical vocabulary intended for those who deploy these technologies and those who approve budgets. For each term I answer: what is it (software, artifact, or practice)? what job does it do? and which kind of output does it serve? To explain the last point, first we distinguish two output types.
Deterministic vs probabilistic outputs
A deterministic output returns the same answer every time you ask the same question. "What was ARR for the last twelve months?" should yield an identical number whether it’s requested via dashboard, API, Excel, or an AI agent. Deterministic outputs have traceable lineage: you can point at the calculation and walk someone through how the number was produced.
A probabilistic output comes from systems that are non‑deterministic by design. Change the question slightly—"How can we improve ARR over the next twelve months?"—and the same model can give different answers on successive runs. That variability is inherent: an LLM predicts likely token sequences across billions of parameters, so outputs vary.
Neither type is superior; both are necessary. The common failure is asking a probabilistic system for a deterministic answer without realizing it. Many of the terms below exist because the industry is trying to place enough structure around probabilistic systems so they can return deterministic answers when required.
Semantic layer
A semantic layer is software that sits between your data and the people and tools that consume it, providing a single place to access trusted, governed metrics. It holds three core things: definitions (how we calculate a business metric), context (what a model or column contains and how it’s used), and relationships (how data fits together). Modern tools often include query engines, caching, and centralized access control, but definitions, context, and relationships are the core.
Why it matters for AI: the semantic layer lets an agent navigate data instead of reasoning over it. Without it, an agent asked for last year’s ARR has to inspect table names, guess joins, infer which date field represents revenue recognition, and reconstruct business logic that lives in someone’s head—probabilistic reasoning that can yield different answers day to day. With a semantic layer, the agent looks up ARR, queries the definition, and returns the same number every time—a deterministic answer delivered through a probabilistic tool.
Gartner now predicts that universal semantic layers will be treated as critical infrastructure by 2030, alongside data platforms and cybersecurity.
Ontology
"Ontology" is the term most likely to derail a meeting. Palantir popularized it commercially, but the underlying idea comes from decades of academic work on formally describing things and their relationships.
A practical distinction: a semantic layer answers "what does this number mean and how is it calculated?" An ontology answers "what things exist in this business and how do they relate to each other?" The semantic layer is metric-first; the ontology is entity-first—customer, order, shipment, facility, supplier, and the rules governing them.
Overlap exists—both encode relationships—and vendors increasingly ship both capabilities under one product name, which blurs terms. Practically, if you need consistent numbers across reporting tools, the semantic layer is central. If you need an agent that reasons about business objects and acts on them, an ontology supplies the model of the world to act in.
One clarification: an ontology is not software. It’s a model or artifact your organization authors and maintains. Software delivers it; the value is in the modeling.
Knowledge graph
If an ontology is the schema, a knowledge graph is that schema populated with real data. The ontology says a customer places an order; the knowledge graph holds your real customers, your real orders, and the edges connecting them, stored as nodes and relationships instead of rows and columns.
When to use a graph instead of a semantic layer? Warehouses and semantic layers excel at aggregation (how much, how many, when). Graphs excel at connection (what is linked to what, and how far apart). Questions like "Which suppliers are two steps removed from this delayed shipment?" or "Which accounts share a beneficial owner?" are graph questions. You can answer them with SQL, but it’s painful.
Graph traversal is deterministic: given the same graph and query, you get the same path every time, which makes graphs valuable grounding for agents. Relationships are modeled facts, not inferences made at inference time.
A knowledge graph does not replace a semantic layer—mature architectures increasingly run both.
Context
"Context" is heavily overloaded and should be split:
-
Deterministic context: metadata stored in your semantic layer or ontology—field descriptions, metric definitions, object relationships, business rules. Metadata used to be human-facing documentation that grew stale; now agents read it at query time, making it functional infrastructure: versioned, reviewed, and consistent.
-
Runtime context: what an agent assembles at the moment of inference—system prompts, conversation history, retrieved documents, tool outputs. Ephemeral and directly influences answers: same question, different context window, different output.
Also: structured context (governed data) vs unstructured context (policy PDFs, contracts, tickets, wiki pages). Unstructured context is usually retrieved by similarity search and thus brings probabilistic behavior: what surfaces depends on how the question is phrased.
Practical rule: when someone says their tool is "context aware," ask which kind. Deterministic context makes answers repeatable; runtime context makes them relevant. Confusing the two leads teams to trust answers that were only true for one prompt.
Observability
Observability is telemetry that tells you whether your systems are still doing what you expect. It’s not data quality (a judgment about correctness) nor testing (checks for anticipated failures). Observability is instrumentation that lets you ask "is this still working?" without predicting exactly how it might break.
On the deterministic side: freshness, row counts, schema changes, null rates, job failures, and lineage impact. If ARR should refresh at 6 a.m. and it didn’t today, you want to know before the CFO does.
On the probabilistic side it’s harder: a system can produce a plausible, fluent answer that is nonetheless wrong. Monitoring here includes evaluation sets scored over time, tool call success rates, retrieval relevance, refusal and fallback rates, latency, cost per query, and structured human feedback.
Drift occurs when the world changes under a running system: data drift (input shifts—new business units, volume changes, unexpected nulls) and model drift (behavior shifts—new model versions or prompt changes). Deterministic systems tend to fail loudly; probabilistic systems fail quietly. Observability is how you catch the quiet failures.
The working glossary
- Deterministic output: the same answer to the same question every time, with traceable calculation.
- Probabilistic output: different answers to the same question across runs, produced by prediction.
- Semantic layer: software storing definitions, context, and relationships behind business metrics and serving them consistently to downstream tools.
- Ontology: a model of business objects, their relationships, and governing rules (an artifact, not software).
- Knowledge graph: an ontology populated with real data as nodes and edges so systems can traverse relationships instead of reconstructing them via joins.
- Context: information needed to use data correctly—either governed in a semantic model or assembled at runtime by an agent.
- Observability: telemetry that tells you whether your data and AI systems are still doing what you expect.
Read that list in order and a pattern emerges: these are layers, not competitors. The ontology describes existence; the knowledge graph holds instances; the semantic layer defines measures; context is how the model receives information; observability tells you when something stops working. Terms feel like they fight because vendors often sell them as substitutes, while in practice they stack.
This vocabulary will evolve. Two years from now some words may be absorbed into product names and shift in meaning. That’s fine, provided your team shares answers to two questions about any term presented to them: what is it actually (software, artifact, or practice)? and which kind of output does it serve (deterministic or probabilistic)? Those two questions cut through most of the noise—agree on the words first, and architecture debates get much shorter.
