Ungoverned RAG Hold

Overview

Ungoverned RAG means indexing documents, tickets, emails, wiki pages, code, or database exports into a retrieval layer without preserving source permissions, ownership, freshness, provenance, retention, quality checks, or incident response. The pattern is attractive because retrieval-augmented generation can make enterprise knowledge feel searchable quickly, but the same pipeline can expose confidential content, retrieve stale or poisoned documents, and produce confident answers that are difficult to audit. NVIDIA's AI Red Team reports that insecure access control in RAG data sources is a recurring issue, including source permissions that are not maintained, RAG stores that fail to reproduce source-specific permissions, overpermissioned read tokens, and permission-propagation delays that leave data exposed (NVIDIA AI Red Team).

The key distinction is not RAG versus no RAG; it is governed retrieval versus ungoverned retrieval. NIST's Generative AI Profile recommends documenting data origin and content lineage, evaluating data and content flows, monitoring privacy risks, reviewing generated sources and citations, and verifying that retrieval-augmented generation data is grounded (NIST AI 600-1). OWASP also warns that sensitive information disclosure can include PII, financial details, health records, confidential business data, credentials, and legal documents, and recommends least-privilege access, secure runtime orchestration of external data sources, retention policies, and data sanitization (OWASP LLM02: Sensitive Information Disclosure).

The reason to classify ungoverned RAG as Hold is that it often moves faster than the controls required to make answers trustworthy and safe. RAG does not automatically solve hallucination, prompt injection, access control, provenance, or compliance. OWASP explicitly notes that RAG and fine-tuning do not fully mitigate prompt-injection vulnerabilities, and that malicious content in external sources can alter model behavior when retrieved into context (OWASP LLM01: Prompt Injection).

Adoption Signals

• RAG has become a default enterprise AI architecture because it can incorporate up-to-date external data without retraining the model, but NVIDIA notes that the retrieval step can also become a vector for attacker-injected data (NVIDIA AI Red Team).
• Security guidance has moved from generic LLM cautions to RAG-specific controls. NVIDIA identifies two common RAG weaknesses: per-user read permissions not implemented correctly and broad write access to the RAG data store, which can enable indirect prompt injection, poisoning, and exfiltration chains (NVIDIA AI Red Team).
• Governance frameworks now explicitly include retrieval-augmented generation in GenAI risk management. NIST calls for documentation of RAG approaches, provenance of data sources, evaluation of data/content flows, regular adversarial testing, monitoring for PII or sensitive-data exposure, and review of sources and citations in outputs (NIST AI 600-1).
• RAG evaluation tooling has matured enough that unmanaged quality is no longer defensible. LangSmith frames RAG evaluation around correctness, relevance, groundedness, and retrieval relevance, while Ragas lists metrics such as context precision, context recall, response relevancy, faithfulness, answer accuracy, context relevance, and response groundedness (LangSmith RAG evaluation, Ragas metrics).
• Observability-oriented evaluation practices are emerging. Arize Phoenix separates retrieval evaluation from response evaluation, captures retrieved documents in traces, evaluates document relevance, QA correctness, and faithfulness, and uses those traces to debug whether poor retrieval or generation caused a failure (Arize Phoenix: Evaluate RAG).
• OWASP's LLM guidance makes clear that safe LLM systems need least privilege, external-content segregation, output validation, human approval for privileged actions, and adversarial testing, which are incompatible with a vector store that ignores source policy and runtime user scope (OWASP LLM01: Prompt Injection, OWASP LLM06: Excessive Agency).

Risks

• Permission collapse is the central failure mode. If ingestion uses a broad service account or the vector store cannot reproduce document-level permissions, users may retrieve content they were never authorized to see; NVIDIA calls out overpermissioned read tokens and stale permission propagation as common causes (NVIDIA AI Red Team).
• Stale retrieval creates false authority. RAG answers can cite obsolete policies, old tickets, superseded contracts, or deleted documents unless the pipeline has freshness rules, reindexing SLAs, deletion propagation, and source versioning; NIST recommends tracking dataset modifications, data deletions, provenance, and content lineage across the AI lifecycle (NIST AI 600-1).
• Poisoned content can become an instruction channel. OWASP describes indirect prompt injection as malicious instructions embedded in external sources such as websites or files, and includes a RAG scenario where an attacker modifies a repository document so the retrieved content alters the model's output (OWASP LLM01: Prompt Injection).
• Broad write access turns retrieval into an attack surface. NVIDIA warns that broad write access to the RAG data store can allow targeted indirect prompt injection, application-result poisoning, and later-stage exfiltration of user documents or personal data (NVIDIA AI Red Team).
• Unmeasured RAG quality degrades silently. Without evaluation datasets, retrieval relevance checks, groundedness tests, and response correctness scoring, teams cannot tell whether bad answers come from poor chunking, weak retrieval, stale sources, irrelevant context, or model hallucination; LangSmith and Phoenix both treat retrieval evaluation and response evaluation as separate concerns (LangSmith RAG evaluation, Arize Phoenix: Evaluate RAG).
• Compliance cannot be reconstructed after the fact. If prompts, retrieved chunks, source versions, user identity, authorization decisions, and generated outputs are not logged with appropriate retention and privacy controls, incident forensics and audit evidence are weak. NIST recommends system inventories, provenance metadata, monitoring, incident response procedures, and documentation of generated-content risks (NIST AI 600-1).
• Downstream action risk compounds retrieval risk. A RAG answer that feeds tools, agents, or workflow automation can trigger confidentiality, integrity, and availability impacts if the system has excessive functionality, permissions, or autonomy; OWASP recommends limiting tools and permissions, tracking user authorization scope, enforcing authorization downstream, and requiring human approval for high-impact actions (OWASP LLM06: Excessive Agency).

Recommendation

Hold on RAG implementations that index everything into a vector store without permissions, freshness, ownership, source visibility, retention rules, or evaluation. Do not connect broad enterprise corpora to a general-purpose assistant until the retrieval layer can answer four questions for every chunk: who is allowed to see it, where it came from, how current it is, and why it was retrieved. If those answers cannot be shown in logs and tested automatically, the system should remain a prototype limited to low-risk, non-sensitive sources.

Use governed retrieval instead. Preserve source-system permissions at ingestion and retrieval time, avoid overbroad service accounts, propagate deletions and permission changes quickly, label source trust levels, separate externally writable content from authoritative corpora, and apply content filtering to retrieved chunks before generation. Treat source metadata, citations, document versions, access decisions, and retrieval traces as first-class production data, not debugging extras.

Build a minimum evaluation and operating model before production use. Maintain golden question-answer datasets, adversarial prompts, poisoned-document fixtures, and regression tests for retrieval relevance, context precision/recall, faithfulness, answer correctness, refusal correctness, and sensitive-data leakage. Assign source owners, define reindexing and retention SLAs, monitor retrieval and output anomalies, rehearse incidents, and block or escalate answers that cannot be grounded in authorized, current, and traceable sources.

Sources

• NVIDIA AI Red Team: Practical LLM Security Advice
• NIST AI 600-1: Generative AI Profile
• OWASP LLM01: Prompt Injection
• OWASP LLM02: Sensitive Information Disclosure
• OWASP LLM06: Excessive Agency
• LangSmith: Evaluate a RAG application
• Ragas: Available metrics
• Arize Phoenix: Evaluate RAG