5.2Code Coverage: What It Measures and What It Doesn't

What Code Coverage Tells You

Code coverage measures which parts of the code are executed when your tests run. It answers the question: "Which lines of code have at least one test touching them?"

Types of Coverage

Type	Measures	Strength	Weakness
Line/Statement	% of lines executed	Easy to understand and collect	A line can be executed without being tested meaningfully
Branch	% of if/else branches taken	Catches missing conditional paths	Does not verify the correctness of each branch
Function	% of functions called	Quick overview of untested functions	A function can be called without its output being verified
Path	% of all possible execution paths	Most thorough	Exponential growth in complex code; often impractical
Condition	% of boolean sub-expressions	Catches complex conditional logic gaps	Difficult to interpret for non-trivial conditions

What Code Coverage Does NOT Tell You

# This function has a bug: it should return a + b, but returns a * b
def calculate_total(a, b):
    return a * b   # Bug!

# This test achieves 100% line coverage but does NOT catch the bug
def test_calculate_total():
    result = calculate_total(2, 3)
    assert result > 0   # Weak assertion! Passes for both + and *

In this example:

• Line coverage: 100% (every line is executed)
• Branch coverage: 100% (no branches to miss)
• Bug detection: 0% (the weak assertion does not verify the correct result)

The lesson: Coverage measures execution, not verification. A test that runs code but does not assert the correct behavior is theater, not testing.

When Coverage Numbers Lie

Scenario	Coverage Says	Reality
Tests with no assertions	High coverage	Zero bug detection
Tests that catch exceptions silently	High coverage	Errors are being suppressed
Tests that test the same code path multiple ways	Very high coverage	Redundant tests, not broader coverage
Generated tests that maximize coverage	90%+ coverage	Tests verify code runs, not that it's correct
Excluding test files from coverage measurement	Artificially high	Denominator is smaller than it should be

Healthy Use of Coverage Metrics

• Use coverage to find blind spots, not to prove quality. "This module has 20% coverage -- we need to investigate" is useful. "We have 90% coverage so the product is ready" is not.
• Track coverage trends, not absolute numbers. Coverage going from 60% to 65% means the team is investing in testing. Coverage stable at 90% while new features are added means new code is untested.
• Require minimum coverage for critical modules: payment processing, authentication, data handling.
• Do not set team-wide coverage targets without context. Requiring 80% coverage for a logging module is wasteful. Requiring 80% coverage for the payment engine is essential.