6.2The Quality Evaluation Checklist

Score each AI-generated test on these six dimensions. A test that fails any dimension should be revised or deleted.

[ ] CORRECTNESS
    [ ] Assertion is actually checking the right thing
    [ ] Expected value matches the specification (not an AI hallucination)
    [ ] Test would fail if the feature broke (mutation test it mentally)

[ ] INDEPENDENCE
    [ ] Test does not depend on execution order
    [ ] Test does not share mutable state with other tests
    [ ] Setup/teardown is self-contained

[ ] DETERMINISM
    [ ] No dependency on current time/date without mocking
    [ ] No dependency on random data without seeding
    [ ] No dependency on external services without mocking

[ ] READABILITY
    [ ] Test name describes the scenario, not the implementation
    [ ] Arrange/Act/Assert sections are clearly separated
    [ ] No "magic numbers" without explanation

[ ] COVERAGE VALUE
    [ ] This test covers a scenario not already covered by another test
    [ ] This test would catch a realistic bug
    [ ] This test is not just asserting that "code runs without error"

[ ] MAINTAINABILITY
    [ ] Uses page objects/fixtures/helpers, not raw selectors everywhere
    [ ] Assertion messages are descriptive
    [ ] Test is under 30 lines (not a novella)

How to Apply the Checklist Efficiently

Do not check every dimension for every test sequentially. Instead, use a three-pass approach:

Pass 1: Scan (2 minutes for 30 tests) Read only the test names. Do they make sense? Do they follow the naming convention? Are there obvious duplicates? This catches about 20% of issues immediately.

# GOOD test names -- clear scenario and condition
def test_should_create_order_when_valid_payload(): ...
def test_should_reject_order_when_quantity_exceeds_maximum(): ...
def test_should_return_404_when_product_not_found(): ...

# BAD test names -- vague or implementation-focused
def test_order_creation(): ...          # What about order creation?
def test_post_request(): ...            # What post request?
def test_validates_correctly(): ...     # Validates what correctly?

Pass 2: Assertions (5 minutes for 30 tests) Read only the assertion lines. Are they checking the right thing? Are they specific enough? This catches tautologies and weak assertions.

# WEAK: only checks status code
assert response.status_code == 200

# STRONG: checks status code AND response content
assert response.status_code == 200
body = response.json()
assert body["name"] == "Widget"
assert body["price"] == 29.99
assert "id" in body

Pass 3: Deep review (10 minutes for 30 tests) For tests that passed the first two passes, check independence, determinism, and setup correctness. This is where you catch the subtle bugs.