RAG in production
Retrieval-augmented generation is easy to demo and hard to productionize. The difference is permission-aware retrieval, grounded answers, and evaluation that catches hallucination.
A RAG demo can take an afternoon. An enterprise-ready system requires more discipline, starting with clear access permissions.
RAG implementation is the most requested capability in enterprise AI right now, because it promises answers grounded in the organization's own knowledge. The promise is real. The failure modes are specific, and avoidable.
Permission-aware retrieval is non-negotiable
The first question of any enterprise RAG system is not "what is the best answer" but "what is this user allowed to know." If retrieval can surface a document the user could not otherwise open, you have built a data-exposure engine with a friendly chat interface. Access control must carry through retrieval: the index respects the same permissions as the source systems, and every query is scoped to the requester's rights.
If retrieval can surface what the user cannot open, you have built a data-exposure engine with a chat interface.
Ground every claim, and show your sources
An answer without citations is an assertion. A production RAG system grounds each claim in retrieved evidence and shows the user where the answer came from. This does two things: it lets the user verify, and it makes hallucination visible. When an answer cannot be grounded, the honest system says so rather than inventing a confident paragraph.
The unglamorous parts decide quality
Teams obsess over which model to use. In practice, retrieval quality is governed by the parts no one demos: how documents are chunked, how candidates are ranked and re-ranked, how freshness and duplication are handled, how metadata is used to filter. Get these right with a mediocre model and you beat a great model fed poor context.
Evaluate grounding explicitly
The metric that matters most for RAG is grounding: what fraction of the answer's claims are actually supported by the retrieved sources. Build this into the evaluation harness. A system that is accurate but ungrounded is a system that is right by luck, and luck does not survive a production audit.
Key takeaways
- Retrieval must carry the user’s access rights: never retrieve what the user cannot see.
- Ground every answer in sources and show the citations; un-cited claims are a defect.
- Chunking, ranking, and freshness decide quality more than the model choice.
- Evaluate grounding explicitly: measure how often answers are supported by retrieved evidence.