An AI answer you can't verify is an opinion with good grammar.
That's the core problem with putting an agent on top of a user's own documents. The whole point is that the user trusts their documents more than they trust the model. So an answer that might come from the documents, or might come from the model's general knowledge, inherits the trust of neither.
I built a RAG pipeline for our production agent that answers from a user's own documents with real citations: every claim traceable to the actual passage it came from. Building it taught me that citations aren't a UI garnish. They change the system's design.
Citations force honesty about retrieval
If the answer must point at a passage, you can't paper over weak retrieval with fluent generation. When retrieval misses, a citation-less system confidently improvises. A citing system visibly comes up empty, which sounds worse but is exactly what you want: a failure you can see is a failure you can fix.
"I don't know" becomes a real answer
With citations, the agent can distinguish "your documents don't cover this" from "here's what they say." That distinction is the difference between a search tool users trust and a chatbot they double-check.
Trust moves on verifiability, not accuracy
A slightly less polished answer with a source the user can click beats a beautiful paragraph they have to take on faith. Users don't measure your retrieval precision. They measure how it felt the one time they checked, and whether the receipt was there.
Grounding beats guessing in exactly the cases that matter
Generic questions are where general models shine. But users bring their own documents for the questions that are specific to them, their contract, their data, their internal doc. Those are precisely the questions where a confident guess does the most damage.
The takeaway
If you're adding document Q&A to a product, my one recommendation: make citations a launch requirement, not a fast-follow. Retrofitting them later means re-architecting retrieval anyway. We built them in from the start, and it's the single design choice users mention most.