64 / 70 · 07 Security Testing for AI Apps · Building a Comprehensive AI Security Testing Program← prev⊞ allnext →☰ Read as one page
11.1The Layered Security Model
A complete AI security testing strategy operates in four layers, each catching different types of vulnerabilities at different stages of the development lifecycle:
+--------------------------------------------------------------------+
| Layer 1: Shift-Left (Every Commit) |
| - Semgrep/CodeQL SAST for AI-specific patterns |
| - Dependency scanning (Snyk/Dependabot) for ML library CVEs |
| - Secret detection (GitLeaks) for API keys in prompts |
| - Unit tests for output sanitization |
+--------------------------------------------------------------------+
| Layer 2: Pre-Production (Every PR/Deploy) |
| - Prompt injection test suite (direct + indirect) |
| - Jailbreak test suite (role-play, encoding, escalation) |
| - Data leakage scanner (PII, system prompt, copyright) |
| - RAG security tests (poisoning, citation accuracy) |
| - OWASP ZAP DAST scan against staging |
+--------------------------------------------------------------------+
| Layer 3: Pre-Release (Before GA) |
| - Red team exercise (human adversarial testing) |
| - Bias and fairness assessment (EU AI Act compliance) |
| - Penetration testing (traditional + AI-specific) |
| - Threat model review |
+--------------------------------------------------------------------+
| Layer 4: Production (Continuous) |
| - Output monitoring (PII scanner on live responses) |
| - Anomaly detection (unusual query patterns, extraction attempts) |
| - Rate limiting and abuse detection |
| - Compliance audit logging |
+--------------------------------------------------------------------+