40 / 87 · 02 AI-Augmented Test Design · Common AI Test Failures← prev⊞ allnext →☰ Read as one page
7.3Failure Mode 2: The Happy Path Only
AI generates 10 tests, all for successful cases. No error handling, no edge cases, no auth failures.
# AI generates these:
def test_create_order_valid(): ...
def test_create_order_with_coupon(): ...
def test_create_order_express_shipping(): ...
def test_create_order_multiple_items(): ...
def test_create_order_with_notes(): ...
# Missing: what happens with invalid product ID?
# Missing: what happens with out-of-stock product?
# Missing: what happens with payment failure?
# Missing: what happens with expired auth token?
# Missing: what happens with zero quantity?
Why AI generates this: LLMs are trained on code where the majority of test examples are happy-path tests. The statistical weight of positive examples outweighs negative ones.
How to detect it: Count the ratio of success assertions to error assertions. If more than 60% of tests assert status_code == 200/201, you have happy-path bias.
How to fix it: Add an explicit prompt instruction:
For every happy-path test you generate, also generate:
- 1 test for missing required fields
- 1 test for invalid field types
- 1 test for authentication failure
- 1 test for authorization failure (wrong role)
- 1 test for resource-not-found