RAG without leaking your data
Retrieval-augmented generation puts your internal documents in front of a model. Here is the access control, redaction and logging you need before it reaches production.
The shortcut everyone takes
A RAG prototype is easy: embed the company wiki, put a chat box in front of it, demo it on Friday. The problem appears the following Monday, when the sales team asks it a question and it happily quotes a document from the HR folder.
Retrieval-augmented generation does not create a new security model. It **inherits whatever access control you did or did not apply at indexing time** — and most prototypes apply none.
Four failure modes
1. **Flattened permissions.** The index has no notion of who may read what, so every user effectively gets the union of everyone's access.
2. **Personal data in embeddings.** Customer records vectorised into a store that has no deletion path, which makes a GDPR erasure request genuinely difficult to honour.
3. **Prompt injection via documents.** A retrieved document contains instructions. The model follows them. Untrusted content is now an instruction channel.
4. **Unlogged retrieval.** You can see the answer but not the sources, so you cannot audit or explain any given response afterwards.
The controls that matter
**Filter at query time, not just at index time.** Store the source document's access control list as metadata on every chunk, and apply the caller's identity as a hard filter on the retrieval query. Post-filtering results after the model has already seen them is not access control.
**Redact before you embed.** Run a detection pass for personal and payment data on ingestion. For most internal knowledge bases the correct action is to strip it — the answer quality loss is negligible and the compliance exposure drops sharply.
**Keep a deletion path.** Map every chunk back to a source document ID, so a deletion request removes the source, the chunks and the vectors in one operation. Test this before launch, not after the first request arrives.
**Treat retrieved text as untrusted.** Fence retrieved content clearly in the prompt, instruct the model that it is reference material rather than instructions, and validate outputs against expected structure. This mitigates rather than eliminates injection, so combine it with output guardrails.
**Log the retrieval, not just the answer.** For every response, store the query, the retrieved chunk IDs, the model, and the user. This is what turns "the AI said something wrong" from an unanswerable complaint into a five-minute investigation.
Residency and vendor terms
If your documents include EU personal data, confirm three things in writing: where the embedding model runs, where the vector store persists data, and whether the provider trains on your inputs. A gateway that routes model calls centrally makes this enforceable rather than aspirational, because you can pin regions and providers in one place instead of in every service.
A reasonable rollout order
Start with a single, low-sensitivity corpus — public product documentation is ideal. Add identity filtering before you add the second corpus. Add redaction and retrieval logging before any corpus containing customer or employee data. The controls are much cheaper to add in that order than retroactively.
Guardrails, retrieval logging and model routing are exactly what bluebill's AI governance work puts in place, so teams can ship RAG without turning it into an open question at the next audit.