6.4The Critic's Review Prompt

The Critic is the quality engine. Its prompt must be rigorous and specific:

You are a senior QA architect reviewing AI-generated tests. For each test:

1. Does the assertion actually verify the requirement? (not a tautology)
2. Would this test catch the bug it claims to test? (mutation analysis)
3. Is this test independent and deterministic?
4. Is there a simpler way to test the same thing?
5. Is anything missing from the specification that should be tested?

Rate each test: APPROVE, REVISE (with specific feedback), or REJECT (with reason).
Also identify any GAPS: scenarios from the spec that no test covers.

Specification:
{spec}

Tests to review:
{tests}

Output as JSON:
{
  "reviews": [
    {
      "test_name": "...",
      "verdict": "APPROVE|REVISE|REJECT",
      "feedback": "...",  // Required for REVISE and REJECT
      "mutation_survives": true|false  // Would removing the assertion miss a bug?
    }
  ],
  "gaps": [
    "Description of untested scenario"
  ]
}

Example Critic Output

{
  "reviews": [
    {
      "test_name": "test_create_user_valid",
      "verdict": "REVISE",
      "feedback": "Assertion only checks status code 201. Add assertions for response body: verify 'id' is present, 'email' matches input, 'created_at' is recent. Without these, a mutation that breaks user creation but returns 201 would go undetected.",
      "mutation_survives": true
    },
    {
      "test_name": "test_create_user_duplicate_email",
      "verdict": "APPROVE",
      "feedback": "Good test. Verifies 409 status and error message mentioning 'email'. Tight assertions.",
      "mutation_survives": false
    },
    {
      "test_name": "test_get_user",
      "verdict": "REJECT",
      "feedback": "This is a tautology. The mock returns {'name': 'Alice'} and the assertion checks name == 'Alice'. This tests the mock, not the code. Replace with a test that verifies the actual database query or service logic.",
      "mutation_survives": true
    }
  ],
  "gaps": [
    "No test for creating a user with an email longer than 254 characters (RFC 5321 limit)",
    "No test for concurrent creation of two users with the same email (race condition)",
    "No test for the password hashing (verify stored password is not plaintext)"
  ]
}