10.1ADR-001: Agent-Driven vs Traditional Test Execution

Context

Traditional test frameworks (Selenium, Playwright, Cypress) execute tests as deterministic scripts. AI agents introduce non-deterministic reasoning into the test execution loop.

Decision

Use an agent-as-orchestrator model: the AI agent reads test definitions (natural language or structured), decides how to interact with the application, and reports results. The framework provides the tools (the Playwright CLI), but the agent decides the execution strategy.

Consequences

• (+) Tests are more resilient — the agent can reason about unexpected states
• (+) Test definitions can be higher-level ("verify login works") instead of step-by-step
• (+) Self-healing: when the UI changes, the agent re-snapshots and adapts (and Playwright's healer agent productizes this loop — see the Self-Healing Strategies chapter)
• (-) Non-deterministic — the same test might execute differently each time
• (-) Harder to debug — agent reasoning is opaque compared to line-by-line scripts
• (-) Slower per-test than deterministic execution

Mitigation

• Log every command the agent executes (for reproducibility)
• Screenshot on every state change (for debugging)
• Set deterministic timeouts (prevent infinite loops)
• Archive Playwright traces where available — the flight recorder for what the agent did
• Fallback to deterministic scripts (generated .spec.ts) for critical paths