A common production story: you spend weeks tuning an AI chatbot, its answers check out, stakeholders approve and you deploy. Three months later the system is confidently wrong about roughly one third of user queries. The model and prompts haven’t changed. The external world has — pricing, policies, or product specs changed — but the underlying knowledge store didn’t keep up.
This isn’t hypothetical. It’s one of the most frequent failure modes in enterprise AI today, and most data engineering teams lack the tooling to detect it, regardless of how the AI retrieves data.
A failure that doesn’t look like a failure
An AI application does not distinguish whether it’s reading from a vector store, a document index, or an API: standard retrieval pipelines rarely check whether the served information is still correct. A stale pricing document can score as highly relevant as a current one, because the system ranks on relevance or availability rather than on factual correctness. A record with a silently missing field passes through for the same reason.
Thus the failure is invisible by design: outdated or incomplete data still scores high, dashboards stay green, and the system appears to operate normally — while providing wrong answers with high confidence.
I observed a similar pattern in a fintech pipeline: an upstream system changed a field without notifying downstream consumers. The pipeline didn’t fail technically; it simply propagated bad values into dashboards because it only checked whether the job completed, not whether the data remained correct. The issue surfaced when a customer noticed an inconsistency — by then the bad data had already spread downstream.
Whether it’s an out-of-date document or a quietly missing field, the symptom is the same: absence of an error does not equal presence of correctness. Without proper validation layers, nothing in the pipeline will identify the problem.
Why this is a data engineering problem
Teams that confront this failure often misdiagnose it — and then repeat the same mistake.
Blaming the model: the first instinct is to blame the model, try another LLM, or tweak prompts. The real problem sits upstream at the data engineering layer: monitoring built for pipeline health, not for data correctness.
Blaming the retrieval layer: once the model is ruled out, teams often blame the retrieval or context layer and buy a different vendor solution. That reaction is predictable: as enterprises push agents into production, the gap between business reality and the data feeding agents becomes visible, and vendors respond. For example, AWS entered the “context layer” race with a knowledge graph that learns from agent usage, and Snowflake’s Horizon Context and Cortex Sense target the issue of agents giving confident but wrong answers because nothing governs the business logic underneath.
Those vendor responses address a real problem, but they often operate one layer above the root cause: a knowledge graph still depends on whatever feeds it. The central issue remains upstream: teams check whether jobs ran, not whether the data they moved remains true.
What’s actually missing: data observability
Data observability is a familiar concept, but its practical implementation is often incomplete. The key metric isn’t a single percentage; it’s coverage: what fraction of critical datasets have lineage and metadata that are actually queryable, versus existing only in someone’s head.
Concrete examples:
- Uber built a dedicated data quality and observability platform long before retrieval-augmented generation gained attention. Their Unified Data Quality platform supports more than 2,000 critical datasets and detects about 90% of data quality incidents before they reach downstream consumers.
- Netflix solved a different part of the problem by building a company-wide data lineage system so anyone can answer where a dataset came from and what touched it along the way. That system maps dependencies across Kafka topics, ML models, experimentation runs and not just warehouse tables.
Between those examples, two of the four essential capabilities are illustrated. I think of the full solution as four measurable dimensions you should build for:
-
Correctness: Does each record conform to expected schema and rules — correct field types, no unexpected nulls, values within allowed ranges? Tools like Great Expectations and Soda automate row- and column-level validation so you validate proactively rather than remediating after failures. Track the percentage of records passing validation per run.
-
Freshness: Is the data current relative to its source, not just current as of the last check? Track time since last successful update per source and set SLAs per dataset rather than a single blanket threshold; some sources need hourly refresh while others do not.
-
Consistency: Does the same fact read the same way everywhere it’s stored or indexed? This kind of failure is silent and only shows up when two systems that use the same source disagree. Periodic cross-checks between downstream destinations and alerting on mismatch rates above a threshold catch this early.
-
Lineage: Can you trace any output back to its origin and every transform it passed through? This is the precise question Netflix set out to solve.
None of this requires infrastructure beyond what many data teams already have; I know because I’ve built such systems.
At Socure, client data arrived in whatever shape clients provided and sometimes contained silent errors. The challenge was to detect incorrect data before it propagated. The same principles applied: validate incoming data, know its origin, and prevent bad data from becoming someone else’s problem.
Great Expectations became part of that foundation: schema and range validation at ingestion, per-source freshness SLAs, cross-system consistency checks, and file-level lineage. All of it fit into a write-audit-publish pattern: data landed in staging, validations ran, and only data that passed the checks moved downstream.
The result manifested downstream: improved accuracy in reporting, ML models, and the AI retrieval layers built on that data.
What to do Monday morning
If you run retrieval-based AI systems in production, the diagnostic question is not which model to try next or which retrieval architecture to migrate to. Instead, answer these four specific questions:
- Is the underlying data validated against the standards required by its consumers?
- What is the oldest piece of content currently being served with high confidence?
- Could two chunks from the same source disagree in the same retrieval result?
- Could you trace where a given result came from if it turned out to be wrong?
If you cannot answer these reliably, the gap lies between your source systems and whatever your agent reads from. That’s a data engineering fix, not a model swap or vendor migration.
Whether you’re building reporting pipelines, ML systems, or AI agents, correctness, freshness, consistency and lineage make data trustworthy. AI simply exposes weaknesses that have long existed in data engineering.
