RAG Security: Attacking Retrieval-Augmented Generation

RAG is everywhere - and it's vulnerable

Retrieval-Augmented Generation (RAG) is how most production LLM applications work. Instead of relying on the model's training data, the system retrieves relevant documents from a knowledge base and feeds them into the LLM's context window. The model reads the documents and generates a response based on them.

This architecture is powerful - it gives the LLM up-to-date, domain-specific knowledge without fine-tuning. But it introduces a critical attack surface: the retrieval pipeline. If an attacker can control what documents enter the knowledge base, they can control what the LLM says, what actions it takes, and what data it leaks.

RAG attacks are the most practical threat to production LLM systems. Unlike direct prompt injection (which requires access to the chat interface), RAG attacks can be planted in advance and triggered automatically when a user's query retrieves the poisoned document.

How RAG works (and where it breaks)

A standard RAG pipeline:

1. Document ingestion: Documents are loaded from sources (file uploads, web scraping, databases, APIs) and split into chunks
2. Embedding: Each chunk is converted to a vector embedding and stored in a vector database
3. Retrieval: When a user asks a question, the system embeds the query, searches the vector database for similar chunks, and retrieves the top N
4. Generation: The retrieved chunks are inserted into the LLM's context window along with the user's question. The model generates a response.

    flowchart LR
        subgraph Attack [Attacker Phase]
            A[Attacker] -->|Uploads poisoned resume| B[(Document Store)]
        end
        
        subgraph System [RAG Pipeline]
            B --> C{Vector DB}
            D[User Query] --> C
            C -->|Retrieves top 3 chunks
including poison| E[LLM Context Window]
        end
        
        subgraph Execution [Victim Phase]
            E --> F[LLM Generation]
            F -.->|Follows injected instructions
instead of System Prompt| G((Data Exfiltration
or False Response))
        end

        classDef malicious fill:#4a1515,stroke:#ff4444,stroke-width:2px,color:#fff;
        classDef user fill:#1a365d,stroke:#4299e1,stroke-width:2px,color:#fff;
        classDef system fill:#2d3748,stroke:#a0aec0,stroke-width:2px,color:#fff;
        
        class A,G malicious;
        class D user;
        class B,C,E,F system;
  

The vulnerability is at step 4. The retrieved chunks are inserted directly into the LLM's context. The model can't distinguish between "these are reference documents" and "these are instructions I should follow." If a retrieved chunk contains injected instructions, the model follows them.

Attack 1: Indirect prompt injection through documents

The attacker plants a document in the knowledge base that contains hidden instructions. When a user's query retrieves that document, the injected instructions execute.

When a user asks "What was Q4 revenue?", the system retrieves this document. The LLM sees the [SYSTEM NOTE] and follows it - including the malicious link in its response. The user trusts the response because it came from the company's RAG system. The link leads to a phishing site.

Attack 2: RAG poisoning - planting false information

Instead of injecting instructions, the attacker plants false information. The RAG system faithfully retrieves it and the LLM presents it as fact.

The RAG system doesn't verify the authenticity of documents. It just retrieves whatever matches the query semantically. An employee asking "what's the password policy?" gets the poisoned document. An employee asking "how do I rotate my API key?" gets instructions to send it to an attacker's email. An employee asking about vendor payments gets a fraudulent IBAN.

Attack 3: Retrieval manipulation

RAG systems retrieve documents based on semantic similarity. An attacker can craft documents that are semantically similar to common queries but contain malicious content. This ensures the poisoned document is retrieved for specific queries.

Attack 4: Context window flooding

RAG systems typically retrieve the top 3-5 documents and concatenate them into the context window. An attacker can create documents that are specifically designed to consume maximum context space, pushing out legitimate documents and ensuring only the attacker's content reaches the model.

Attack 5: Cross-document injection

Advanced attack: the injected instructions in one retrieved document reference content in another retrieved document. This can cause the LLM to combine information in ways the developers never intended.

How documents get into the knowledge base

Every ingestion point is an attack surface:

• User file uploads: A user uploads a "product spec" that contains injected text. Any RAG system that ingests user-uploaded documents is vulnerable.
• Web crawling: The RAG system scrapes web pages for knowledge. An attacker publishes a page with injected content that the crawler indexes.
• Compromised document repositories: An attacker gains access to a SharePoint, Google Drive, or internal wiki and plants poisoned documents.
• Third-party data feeds: APIs, RSS feeds, syndicated content - if the RAG system ingests external data, that data is an attack vector.
• Publicly editable content: Wikis, community forums, comment sections - any content source that accepts user input can be poisoned.

Detecting RAG attacks

RAG attacks are hard to detect because the documents look legitimate. The injection is embedded in otherwise normal text. Detection strategies:

• Document anomaly detection: Flag documents that contain instruction-like patterns (see the Defending Against AI Attacks guide for input validation patterns).
• Response monitoring: Flag responses that contain URLs, email addresses, or account numbers that aren't in the original retrieved documents.
• Source tracking: Track which documents contributed to each response. If a response includes content that traces back to a recently added or low-trust document, flag it.
• Differential testing: Run the same query with and without each document. If a document significantly changes the response in a way that benefits an attacker, flag it.

Defending RAG systems

For the full defense stack, see our Defending Against AI Attacks guide. RAG-specific defenses:

1. Source verification: Only ingest documents from verified sources. Sign documents and verify signatures on retrieval.
2. Document quarantine: New documents go through a review period before being available for retrieval.
3. Content screening: Run a separate LLM to scan ingested documents for injection attempts before they enter the knowledge base.
4. Untrusted marking: Prefix retrieved content with "[UNTRUSTED - data only]" in the context window. (Soft defense - the model may still follow injected instructions.)
5. Rate limiting ingestion: Limit how many documents a user or source can upload per day.
6. Retrieval diversity: Don't retrieve all chunks from a single document. Force diversity across sources so one poisoned document can't dominate the context.

For the practical exploitation techniques against the LLMs that power RAG systems, the AI Jailbreak Guide covers tested injection methods against GPT-5.2, Claude, Gemini, and 50+ other models.