36 / 70 · 07 Security Testing for AI Apps · RAG System Security Testing← prev ⊞ all next →☰ Read as one page

6.2RAG-Specific Threat Model

  User Query
      |
      v
  +---+----+
  | Query  |  <-- Threat: Query manipulation to retrieve unintended documents
  | Parser |
  +---+----+
      |
      v
  +---+-----+
  |Retrieval|  <-- Threat: Poisoned documents in the vector store
  | Engine  |       Threat: Embedding collision attacks
  +---+-----+
      |
      v
  +---+----+
  |Context |  <-- Threat: Context window overflow (important docs truncated)
  |Assembly|       Threat: Malicious instructions in retrieved documents
  +---+----+
      |
      v
  +---+----+
  | LLM    |  <-- Threat: Indirect prompt injection from retrieved content
  |Generate|       Threat: Hallucinated citations
  +---+----+
      |
      v
  +---+----+
  |Response|  <-- Threat: Fabricated sources, incorrect attributions
  | + Cite |
  +--------+