9.3Part 2: Prompts Are Code -- Test Them Like Code

Your prompt templates determine the quality of every suite the team generates. A silent regression in a template (or a model upgrade that changes behavior under the same template) degrades every future generation session. The fix is the same discipline you apply to source code:

• Version templates in the repo, change them via PR (covered in the prompt templates chapter).
• Review prompt changes as behavioral changes -- a template edit that drops the "include negative cases" instruction is a coverage regression waiting to happen.
• Test templates against golden inputs: a fixed spec excerpt with known expected properties of the output.

A prompt regression test does not assert exact output (LLM output varies); it asserts measurable properties of the output:

def test_api_template_generates_negative_cases(llm, golden_spec):
    """The api-schema-to-tests template must produce error-path tests."""
    suite = llm.generate(template="api-schema-to-tests", artifact=golden_spec)

    tests = parse_test_functions(suite)
    negative = [t for t in tests if asserts_error_status(t)]

    assert len(tests) >= 15, "Template output volume dropped"
    assert len(negative) / len(tests) >= 0.35, "Happy-path bias regression"
    assert all(has_assertions(t) for t in tests), "Assertion-free tests appeared"

Run this on every template change and after every model version bump. This is your first taste of eval thinking: probabilistic output, property-based assertions, thresholds instead of exact matches.