Many corporate knowledge programs begin the same way: a senior engineer announces she will leave in six weeks and is asked to document the processes she has run for years. The usual deliverable is a clean flowchart of the happy path — accurate and perhaps elegant, but missing the thresholds, observations and situational judgments (for example, which supplier parts fail in humidity) that the veteran has internalized.
Six months later a production line fails and the knowledge base cannot explain what to do. The interview happened, the transcript was stored, chunked, embedded and indexed, and a search returns the relevant passage quickly. But that passage does not answer the practical question because no one had asked the engineer to surface the judgment behind the procedure.
That gap limits many enterprise AI programs today: better retrieval and ranking expose what was recorded, but cannot recover expertise that never entered the collection.
Tacit knowledge: forms and challenges
Michael Polanyi formulated the problem durably in 1966 in The Tacit Dimension: "We can know more than we can tell." Competence often rests on skills, perception and judgment that resist full explication.
In organizations tacit knowledge typically appears in three forms:
- Elicitable knowledge: it is unspoken because no one asked the right, precise question, or because the expert assumes others see what she sees.
- Perceptual knowledge: trained attention, such as when an engineer "hears" a bearing fail or a nurse notices a patient looks wrong before a monitor changes.
- Collective knowledge: a team’s habits, standards and shared sense of what counts as a sound decision in that organizational context.
Preventive judgment adds another difficulty: a failure creates tickets, incident reports and message trails, but an experienced operator who quietly avoids a known failure mode creates no record — the useful outcome is the absence of an event, so data pipelines capture no trace of the decision.
The limits of automation and Polanyi’s paradox
Machine learning can infer rules people struggle to state if the model sees enough representative examples. But rare expertise often lacks sufficient examples: a company may have only a handful of unusual incidents and a single person who has learned, over decades, how to read them.
David Autor described this constraint as “Polanyi’s paradox”: many of the tasks hardest to automate depend on rules we can’t state. Contemporary models circumvent the paradox by learning from examples, but that workaround weakens when examples are scarce. Fine‑tuning teaches a model a company’s vocabulary and document formats, but it cannot reconstruct decisions that left no data.
At the same time the economics have shifted: much documented best practice is now embedded in frontier model training data and thus broadly available to competitors. The more explicit knowledge circulates, the more a company’s advantage depends on local judgment: the exceptions, thresholds, relationships and practiced responses its people have accumulated.
Therefore elicitation becomes an architectural concern rather than just an offboarding chore: the organization needs a repeatable way to surface expressible knowledge, a route for expertise that must be demonstrated, and the humility to distinguish the two.
A protocol for elicitation
The central design question is simple: which follow‑up will prompt an expert to say the missing judgment aloud? The interview quality sets the ceiling for the knowledge base; the index determines how quickly someone reaches that material.
Expert explanations tend to become vague in four places. An effective interview protocol asks targeted questions about each:
- Thresholds: which number, reading or condition triggers action?
- Exceptions: when does the documented procedure stop applying?
- Evidence: what did the expert observe before reaching the conclusion?
- Escalation: who becomes involved, and at what point?
These questions uncover operational details that runbooks often lack. A language model can play a narrow role during the interview: proposing the next question that converts a general statement into a usable rule. The author notes an open source toolkit, ExpertTrace, built around this protocol.
Compare a typical initial answer:
"We review high‑risk use cases before deployment. If the risk seems significant, we escalate to the governance council."
That statement is indexable and retrievable, but a new employee still cannot act. "Seems significant" supplies no decision criterion. A focused follow‑up can yield a much more actionable reply:
"Escalation to the council is required when the use case touches employment, credit, or health decisions, or when model output reaches a customer without human review. Predeployment review is skipped for internal‑only tools with no personal data, which is the exception people most often get wrong. If we cannot identify a named accountable owner, the review does not proceed, regardless of risk tier."
The second answer costs little extra time yet contains a decision rule, an exception, a recurring failure pattern and a blocking condition. It can guide a real dispute rather than merely mentioning the subject.
The protocol needs guardrails: limit follow‑ups so the expert is not exhausted into agreeable noise; keep the model focused on generating questions and separate question generation from compiling and validating answers. An expert’s statement should be recorded with provenance and context; accuracy must be independently reviewed.
A four‑plane knowledge architecture
Elicitation is only part of a larger system. A tacit‑aware architecture has four planes — capture, representation, serving and transmission — and each plane addresses a different failure in the movement of expertise.
-
Capture plane: structured interviews, incident reconstruction, decision journals and observation collect more than polished procedure. Record trigger, evidence, exception and escalation path while the expert can still explain. Route perceptual skill toward demonstration and practice rather than forcing it into prose.
-
Representation plane: preserve the distinctions that make the material trustworthy. A compliance policy, a war story and an untested hypothesis should not become interchangeable chunks. Carry provenance, confidence and validity context (plant, time period, equipment and conditions) as first‑class properties. Extend the knowledge graph beyond documents to the people and episodes that produced them.
-
Serving plane: determine how knowledge reaches users. Answers should cite retrieved evidence and show the source. When the collection cannot answer, the system should say so clearly and route the question to someone with relevant experience. "Ask Joe; she rebuilt this line in 2023" is more useful than a fluent paragraph assembled from weak evidence, and the referral restores the human contact through which difficult knowledge often moves.
-
Transmission plane: help expertise move between people through shadowing, teaching and communities of practice. The platform should detect when knowledge concentration and attrition risk converge, then trigger capture and apprenticeship before a notice period begins.
Gabriel Szulanski’s study of 271 observations of 122 best‑practice transfers across eight companies found that even willing teams struggled to reproduce methods developed elsewhere in the same organization. Difficulty often began with causal ambiguity: people could describe steps without fully understanding why they worked. Receiving teams also needed context and experience to absorb and apply what they learned; preparation, coaching and time helped them rebuild the practice in their own setting. A repository can preserve a record, but receiving teams must turn that record into working knowledge.
Evaluating the knowledge layer
Retrieval precision and answer faithfulness show how well a system serves its existing collection, but not whether the collection contains the knowledge the organization actually depends on. That requires a separate evaluation loop tied to capture priorities and transfer outcomes.
The loop proceeds as follows:
-
Incident replay: select 20–30 resolved incidents, remove the resolutions and give the opening facts to the system. Ask the engineers who solved them to grade its responses. Compare those answers with responses from a frontier model that lacks access to the company’s collection; the gap reveals the generic‑answer rate — how often the internal system merely restates public knowledge.
-
Bus‑factor audit: test questions that only one or two employees can answer and study how the system fails. A clear admission of uncertainty followed by a useful referral is healthy; fluent boilerplate damages trust in every response.
-
Abstention calibration: measure whether the system answers when evidence exists and declines when the corpus cannot support an answer. Build a labeled set of answerable and unanswerable questions and track abstention precision and recall as the collection grows. A system that never says "I don’t know" is unevaluated on the dimension that matters most.
-
Transfer outcomes: measure whether knowledge reached the people who need it. Evidence of transfer appears in shorter time to proficiency, fewer repeat incidents after elicitation, and fewer critical responsibilities that depend on a single person. Document and query counts describe activity; they do not show whether someone else can now make the decision.
Conclusion
A robust knowledge system records what an expert said, preserves the conditions around the statement and marks uncertainty. It also recognizes expertise that requires demonstration, apprenticeship or team practice. Because the people who carry that knowledge go home each evening, the architecture should be ready long before someone gives notice.
