The failure nobody sees coming
Ask a general-purpose assistant what your company’s notice period is and it will tell you. Fluently. With the right tone, the right structure, and a number that sounds exactly like a notice period. It may even be correct — statistically, something close to that number is common.
That is the problem. A wrong answer that looks wrong gets caught in a second. A wrong answer that looks right gets forwarded, pasted into a contract, quoted to a client. The cost of AI inside a company is almost never the subscription. It is the one confident answer a month that nobody thought to verify, because there was nothing about it to verify.
So the question to ask of any assistant you put in front of your team is not "how good are its answers." It is: what does it do when it has no basis for one?
Grounding: answer from what you actually have
A general model answers from everything it has ever read. A grounded assistant answers from a defined set of your own material — your handbook, your contracts, your procedures, your price lists — and nothing else.
That constraint sounds like a limitation. It is the entire product.
The difference shows up immediately. Ask both the same question about your return policy. The general model gives you a sensible industry answer. The grounded one gives you your policy, or it tells you your policy does not cover that case. Only one of those is useful, and it is the one that sometimes says less.
Grounding also changes what a wrong answer means. An ungrounded model that is wrong is wrong invisibly — the reasoning is inside the model. A grounded model that is wrong points at a document, and you can open that document and see that it is out of date. The error becomes findable. That is a different class of system.
A system that can be wrong in a way you can trace is safer than a system that is usually right for reasons you cannot inspect.
Citations are not a footnote, they are the interface
Plenty of tools now add a small "source" link under the answer. Treating it as decoration misses what it does.
A citation changes the user’s behaviour. When the source is visible, people click it on the answers that matter and skip it on the ones that don’t — which is exactly the right allocation of human attention. When there is no source, people either verify nothing or verify everything, and both are bad outcomes.
A citation also changes the builder’s behaviour. Once every answer has to point at a document, you can no longer paper over a thin knowledge base with a fluent model. Gaps in the content become visible as gaps in the answers. The assistant turns into an audit of your own documentation — usually an uncomfortable one, and always a useful one.
Make the source specific. "From the employee handbook" is not a citation; the section, the version and the date are. If the answer came from a document last updated three years ago, the person reading deserves to know that before they act on it.
Permissions are part of the answer, not a setting
An assistant that can read everything will eventually tell someone something they should not have. Not maliciously — because a question was phrased in a way that pulled salary data, or a draft contract, or a document from a department the person has nothing to do with.
The fix is not a filter bolted on afterwards. It is to make access the ground the answer stands on: the assistant retrieves only from what the person asking is already allowed to open, so two colleagues can ask the identical question and correctly receive different answers. If your company already has an org chart and document permissions, that structure should drive the assistant. If the assistant needs its own separate permission model, you now have two systems to keep in sync, and they will drift.
Make the refusal rate a number you watch
Here is the metric almost nobody tracks: how often does the assistant decline to answer?
If it is near zero, the system is not being honest — no document set covers everything people ask. If it is very high, the knowledge base is too thin or retrieval is failing. Somewhere in between is a system telling you the truth about what it holds.
Better still, treat every refusal as a signal rather than a dead end. A question the assistant could not answer is a question your documentation does not cover. Log those, group them, and you have a ranked list of what to write next — generated by the actual needs of the people doing the work, not by whoever had time to write a policy.
Seven questions to ask any vendor
Before you sign anything, make the demo answer these:
- Ask something your documents genuinely do not cover. Does it say so, or does it produce something plausible?
- Ask something your documents contradict themselves on. Does it surface the conflict, or silently pick one?
- Click the source on three answers. Does it land on the exact passage, or just the file?
- Ask the same question as two people with different access. Do the answers differ correctly?
- Ask in one language about a document written in another. Most real companies run on mixed-language documents.
- Ask about something that changed recently. Does it use the current version, or the one it indexed first?
- Ask where the data is processed and stored. The answer should be a sentence, not a meeting.
A vendor confident in their system will enjoy questions one and two. Watch for the ones who steer the demo away from them.
Why we build it this way
We put this principle at the centre of the internal workspace we are building: an employee asks in plain language, the answer comes only from the company’s own documents, the source is shown, and when there is no source the assistant says it does not know rather than filling the gap.
That last behaviour is the hardest one to ship, because it makes a demo look worse. An assistant that answers everything feels impressive for ten minutes. An assistant that occasionally says "I don’t know, and here is what I searched" earns something an impressive demo never does: people start believing it when it does answer.
That trust is the whole asset. Everything else is retrieval.