29 / 62 · 22 Test Strategy & Quality Metrics · Essential QA Metrics← prev ⊞ all next →☰ Read as one page

4.3Test Metrics

Test Coverage

What it measures: How much of the software is exercised by tests.

Types of coverage:

Type	What It Measures	Formula
Line coverage	% of code lines executed by tests	(Lines executed / Total lines) x 100
Branch coverage	% of code branches (if/else) executed	(Branches executed / Total branches) x 100
Function coverage	% of functions called by tests	(Functions called / Total functions) x 100
Requirement coverage	% of requirements with at least one test	(Requirements with tests / Total requirements) x 100
Risk coverage	% of high-risk areas with adequate testing	(Covered risk areas / Total risk areas) x 100

Typical targets:

Unit test line coverage: > 80% (> 90% for critical modules)
Branch coverage: > 70%
Requirement coverage: 100% for critical requirements
Risk coverage: 100% for critical and high-risk areas

Test Pass Rate

What it measures: The percentage of tests that pass in a given execution.

Formula:

Pass Rate = (Tests Passed / Total Tests Executed) x 100

Example:

Nightly regression run:
  Passed: 487
  Failed: 8
  Skipped: 5
  Total executed: 495 (excluding skipped)
  Pass Rate = (487 / 495) x 100 = 98.4%

Why it matters: A consistently high pass rate (> 98%) means the test suite is reliable and the product is stable. A low or volatile pass rate signals either product instability or test suite problems.

Automation Ratio

What it measures: The proportion of test cases that are automated.

Formula:

Automation Ratio = (Automated Test Cases / Total Test Cases) x 100

Interpretation:

Ratio	Interpretation
< 30%	Heavy manual dependency. Release speed is limited by manual execution capacity.
30-60%	Typical for teams actively building automation. Focus on automating the highest-value tests.
60-80%	Good balance. Most regression is automated. Manual testing focuses on exploratory and edge cases.
> 80%	High automation maturity. Remaining manual tests are likely exploratory or require human judgment.

Flaky Test Rate

What it measures: The percentage of tests that produce inconsistent results (pass sometimes, fail sometimes) without code changes.

Formula:

Flaky Test Rate = (Tests with inconsistent results / Total tests) x 100

Why it matters: Flaky tests erode trust in the test suite. When teams cannot trust test results, they stop paying attention to failures, and real bugs slip through.

Typical targets: < 2% is healthy. > 5% requires immediate action.