1.1Why AI Applications Need a New Security Approach
AI applications are still web applications. They still have APIs, databases, authentication systems, and user interfaces. Every vulnerability that exists in traditional web applications -- SQL injection, cross-site scripting, broken access control -- still exists in AI applications.
But AI applications also introduce an entirely new category of vulnerabilities that traditional security testing does not address. When your application includes a Large Language Model, you have added a component that:
Interprets natural language as instructions. Unlike a database that executes only structured queries, an LLM processes free-form text and decides what to do with it. This makes it susceptible to prompt injection -- the attacker's instructions are indistinguishable from legitimate input.
Has access to tools and data. Modern LLM applications give models access to databases, APIs, file systems, and the internet. A compromised model can use these tools to exfiltrate data, modify records, or attack internal systems.
Generates unpredictable output. Traditional applications produce deterministic output for a given input. LLMs produce probabilistic output that can vary each time. This means a security test that passes once might fail on the next run.
May memorize training data. LLMs can inadvertently memorize and reproduce sensitive information from their training data, including PII, API keys, and proprietary content.
Can be manipulated through indirect channels. If your LLM processes external data (emails, documents, web pages), an attacker can embed instructions in that data without ever directly interacting with your application.
The result is a dual attack surface: the traditional web application surface and the AI-specific surface. A comprehensive security testing strategy must cover both.
The Scale of the Problem
Consider a typical AI-powered customer support chatbot:
- It has a system prompt that defines its behavior and restrictions
- It connects to a RAG pipeline that retrieves information from company documents
- It can look up order details via an API
- It can process refunds up to a certain limit
- It maintains conversation history across turns
- Multiple users interact with it simultaneously
Each of these capabilities is an attack vector:
| Capability | Traditional Risk | AI-Specific Risk |
|---|---|---|
| System prompt | N/A | Prompt injection to override behavior |
| RAG pipeline | SQL injection in search | Retrieval poisoning, indirect injection |
| Order lookup API | Broken access control | LLM bypasses access checks via tools |
| Refund processing | Authorization bypass | Social engineering the AI to approve refunds |
| Conversation history | Session hijacking | Cross-session context leakage |
| Multi-user access | Session fixation | Model confusion between user contexts |
A traditional penetration test would catch the traditional risks. It would likely miss every AI-specific risk. This book teaches you to catch both.