105 / 168 · 01 Agent Skills for Browser Automation · Self-Healing Strategies for Agent-Driven Tests← prev⊞ allnext →☰ Read as one page
14.5Self-Healing vs Flaky Tests
What's the Difference?
Flaky test: The test itself is unreliable (race conditions, timing issues, external dependencies). Self-healing doesn't fix this — it masks it.
Broken locator: The application changed, the test's element resolution is stale. Self-healing legitimately fixes this.
How to Tell Them Apart
| Signal | Flaky Test | Broken Locator |
|---|---|---|
| Fails intermittently | Yes | No (consistent failure) |
| Same step works sometimes | Yes | No |
| UI visually unchanged | Yes | No (UI was updated) |
| Error message | Timeout (element exists but timing varies) | Not found (element genuinely missing from snapshot) |
The Rule
Self-healing should fix broken element resolution, not mask flaky tests. If a test needs healing on every run, it's flaky and needs to be fixed at the source. (And if the healer keeps concluding "the app is broken," believe it — that's not a healing failure, that's a bug report.)