12.2What AI Does Well

Codebase and repository analysis — AI excels at digesting large codebases. Given a repository, an LLM can map the architecture, list API endpoints, identify untested code paths, and flag risk areas based on complexity and change frequency. This compresses weeks of manual exploration into hours.

Test generation from specifications — Given a clear spec (API contract, user story with acceptance criteria), AI generates a useful first draft of test cases covering obvious paths. Key word: "first draft." AI-generated tests need human review because they over-test happy paths, under-test edge cases, sometimes hallucinate endpoints/selectors, and lack domain knowledge.

Selector healing — When UI element selectors change (renamed data-testid, updated CSS class), AI-powered tools detect breaks and suggest updated selectors. Narrow but genuinely useful for reducing UI test maintenance.

Coverage gap detection — AI analyzes a codebase and its test suite to identify areas tested poorly relative to their risk, considering complexity, change frequency, and historical bug data.

Regression test selection — AI predicts which tests are likely affected by a code change and recommends running only those. A more sophisticated version of impact-based test selection that improves with historical data.

Interpreting behavioral data — AI is the final, interpretive step of a production-behavior pipeline — never the heavy lifter. Feeding raw logs to a model fails (context limits, token cost, broken sessions). Reduce telemetry to aggregated signals first (raw logs → analytics → aggregations → patterns → LLM → recommendations), then hand the model a 20 KB summary of top journeys, abandonment flows, and incidents. It is good at proposing regression tests, alerts, and exploratory charters — as the analyst reading a summary, not the processor reading the logs.