8.5Prioritizing Test Debt

Not all test debt is equally costly. Prioritize based on impact.

Priority Matrix

Impact	Frequency	Priority	Example
High	High	Fix immediately	Flaky test that blocks PRs daily
High	Low	Fix this sprint	Slow pipeline that frustrates developers
Low	High	Fix next sprint	Test that requires manual data refresh weekly
Low	Low	Backlog	Cosmetic test naming inconsistency

The Flaky Test Problem (Special Section)

Flaky tests deserve special attention because they are the most insidious form of test debt. A flaky test that blocks the pipeline 5% of the time seems minor, but across 50 daily pipeline runs, that is 2-3 blocked pipelines per day. Multiply by the time developers spend investigating and retrying, and a single flaky test can cost hours per week.

Flaky test triage process:

1. Detect: Identify tests that fail without code changes (CI tools like Buildkite and CircleCI track this automatically)
2. Quarantine: Move the flaky test to a separate suite that does not block PRs
3. Investigate: Determine the root cause (timing, state leak, infrastructure dependency)
4. Fix: Address the root cause
5. Restore: Move back to the main suite after verification over 10+ consecutive runs
6. Monitor: Track flake rate trends to ensure the fix holds

Common flaky test root causes:

Root Cause	How to Fix
Timing dependency (sleep, setTimeout)	Use explicit waits for conditions instead of fixed delays
Shared state between tests	Isolate test data; use unique identifiers per test
Order dependency	Run tests in random order to expose dependencies
External service dependency	Mock external services; use stubs
Resource contention	Reduce parallelism; increase timeouts for CI
Date/time dependency	Mock system clock; use fixed dates in tests