40 / 62 · 22 Test Strategy & Quality Metrics · Measuring Test Effectiveness← prev⊞ allnext →☰ Read as one page
5.6Test Suite Health Metrics
Execution Time
Why it matters: If the test suite takes too long, developers stop running it, and the feedback loop breaks.
| Target | Context |
|---|---|
| < 10 minutes | Unit tests (should run on every commit) |
| < 30 minutes | Integration tests (should run on every PR) |
| < 60 minutes | Full regression (should run nightly or per-release) |
Test Stability
Formula:
Test Stability = (Test runs with consistent results / Total test runs) x 100
Target: > 98%. If less than 95% of your test runs produce consistent results, the suite is unreliable and trust in the pipeline will erode.
Maintenance Cost
Track the time spent maintaining tests versus writing new ones:
Maintenance Ratio = Maintenance Hours / Total Test Engineering Hours
Healthy: < 30% (most time spent creating new tests)
Warning: 30-50% (growing maintenance burden)
Critical: > 50% (team is spending more time fixing tests than creating them)