Modern QA2026Failure Mode 4: The Hallucinated API — tiles
Log inJoin
42 / 87 · 02 AI-Augmented Test Design · Common AI Test Failures← prev⊞ allnext →☰ Read as one page

7.5Failure Mode 4: The Hallucinated API

AI invents methods, endpoints, or parameters that do not exist in your codebase.

# AI invents methods that don't exist
result = client.orders.find_by_status("pending")  # find_by_status doesn't exist
user = UserService.get_or_create(email="test@test.com")  # get_or_create doesn't exist
response = client.patch("/api/v2/orders/1/cancel")  # this endpoint doesn't exist

Why AI generates this: The LLM has seen similar APIs in its training data (Django's get_or_create, Rails' find_by_*) and applies those patterns to your codebase.

How to detect it: Always grep your codebase for any function, method, or endpoint the AI references.

# Quick hallucination check
grep -rn "find_by_status" app/ --include="*.py"
grep -rn "get_or_create" app/ --include="*.py"
grep -rn "/cancel" app/routes/ --include="*.py"

If grep returns nothing, the API is hallucinated. Delete the test or replace the call with the actual API.

How to prevent it: Include the actual function signatures or endpoint list in your prompt context.