13.5Failure Analysis

When a test fails, the agent (or framework) should capture:

1. Visual State (Screenshot)

playwright-cli screenshot          # PNG in .playwright-cli/ → copy to failures/${TEST_NAME}/

2. Semantic State (YAML Snapshot)

playwright-cli snapshot            # roles, names, refs → copy to failures/${TEST_NAME}/

The snapshot is the artifact the agent re-analyzes: unlike a raw text dump, it preserves structure ("the error alert exists but is empty" vs "the page contains no error region at all" — different bugs).

3. Location (URL)

playwright-cli eval "location.href" > "failures/${TEST_NAME}/current_url.txt"

4. Console Errors (JavaScript)

playwright-cli eval "JSON.stringify(window.__console_errors || [])" > "failures/${TEST_NAME}/console_errors.json"

5. The Trace (When Available)

For Playwright test-runner executions and healer sessions, archive the trace. It contains every action with before/after snapshots, console output, and — since 1.60 — HAR network capture. It is simultaneously the human debugging artifact (Trace Viewer), the agent debugging artifact (feed it back instead of a vague "it failed"), and the compliance artifact ("show me what the AI did" has a literal answer).

6. Agent Reasoning (Optional)

If running with Claude Code, the agent's reasoning about the failure is captured in conversation history. This can be extracted as a markdown file explaining what happened.