12.3What AI Does Not Do Well

Record-and-replay — Several tools promise to record user actions and generate maintainable tests. The recording works; the "maintainable" part does not. Recorded tests are brittle, tied to specific application state, and expensive to maintain. AI does not fix the fundamental problems with record-and-replay.

Replacing QA engineers — Testing is not primarily a code-generation problem. The hard part is knowing what to test, why to test it, when to test it, and how much testing is enough. These are judgment calls requiring domain knowledge, risk assessment, and business understanding. AI can generate test code; it cannot generate test strategy.

Understanding application behavior — AI can read code and infer what it does syntactically. It cannot know that a subtle layout shift is jarring to users, that an error message is confusing, or that two features interact in unexpected ways. These observations require human judgment and empathy.