14.2The Three Levels of Recovery

Level 1: Re-snapshot and Re-resolve

With ref-driven interaction, the classic "selector broke" failure becomes "ref went stale." The recovery is a fresh snapshot:

Agent: Bash("playwright-cli click e5")
→ Error: ref e5 not found on current page

Agent thinking: "The ref is stale — the page changed since my last snapshot.
Let me look at what's actually there now."

Agent: Bash("playwright-cli snapshot")
→ .playwright-cli/snapshot-014.yaml
Agent reads:
  - button "Cancel" [ref=e7]
  - button "Submit Form" [ref=e8]
  - button "Save Draft" [ref=e9]

Agent thinking: "The submit button is now 'Submit Form' at ref e8.
Same accessible role, same intent."

Agent: Bash("playwright-cli click e8")
→ Success

Token cost: one snapshot read + reasoning (~1,500-5,000 tokens depending on page size) Success rate: high for renames, reordering, and markup refactors — the accessibility tree changes far less often than the DOM

Level 2: Page Analysis with Screenshot + Snapshot

When Level 1 doesn't work, the agent takes a deeper look:

Agent: Bash("playwright-cli snapshot")
→ YAML shows almost nothing: text "Loading… Please wait"

Agent: Bash("playwright-cli screenshot")
Agent: *reads the PNG*

Agent thinking: "There's a loading spinner. The page never finished loading —
this isn't a locator problem at all."

Agent: *waits, re-snapshots — content present now*
Agent: Bash("playwright-cli click e8")
→ Success

Token cost: ~1 screenshot read + 1-2 snapshot reads Success rate: catches timing issues, loading states, redirects, blocking modals

Level 3: Hand the Test to the Healer Agent

When failures persist across runs, stop patching inside the run — this is now test maintenance, and it's what Playwright's healer agent is for:

CI: add-valid-todo.spec.ts FAILED

Healer: *runs the failing test, replays the failing step*
Healer: *inspects the live UI in the actual failing session
        (browser.bind() means it debugs the real state, not a reproduction)*
Healer: *patches the locator, adjusts a wait, fixes test data*
Healer: *reruns until green*

— or —

Healer: "The 'Save' action returns a 500. The functionality is broken.
This test should stay red."

That second outcome is the whole point. A healer that always makes the test pass is a bug-hiding machine. Playwright's framing — "the app is actually broken" is a valid terminal state — is the honesty standard you should hold any self-healing tool to in an evaluation, commercial vendors included.

Cost: a full agent session — expensive, but it produces a durable fix (a patched test in a PR) rather than an in-run workaround Success rate: handles major redesigns; and when it "fails," it has found you a real bug