5.2Pairwise (2-Way) Testing

Pairwise testing ensures that every combination of any two parameters appears in at least one test case. Research consistently shows that most software defects are triggered by interactions between at most two parameters (approximately 70-90% of defects according to NIST studies).

Prompt for Pairwise Generation

Generate a pairwise combinatorial test suite for a user registration form with
these parameters:

- Browser: Chrome, Firefox, Safari, Edge
- OS: Windows, macOS, Linux
- Language: English, Spanish, Japanese
- Account type: Free, Pro, Enterprise
- Auth method: Email/password, Google SSO, SAML

Requirements:
- Every pair of parameter values must appear in at least one test case
- Minimize total number of test cases
- Output as a numbered table

Example Output

A strong LLM produces a near-optimal pairwise set of approximately 16-20 test configurations instead of the full 4 x 3 x 3 x 3 x 3 = 324 combinations:

#	Browser	OS	Language	Account	Auth
1	Chrome	Windows	English	Free	Email/password
2	Chrome	macOS	Spanish	Pro	Google SSO
3	Chrome	Linux	Japanese	Enterprise	SAML
4	Firefox	Windows	Spanish	Enterprise	SAML
5	Firefox	macOS	Japanese	Free	Email/password
6	Firefox	Linux	English	Pro	Google SSO
7	Safari	Windows	Japanese	Pro	Email/password
8	Safari	macOS	English	Enterprise	SAML
9	Safari	Linux	Spanish	Free	Google SSO
10	Edge	Windows	English	Free	Google SSO
11	Edge	macOS	Spanish	Enterprise	Email/password
12	Edge	Linux	Japanese	Pro	SAML
13	Chrome	Windows	Japanese	Pro	SAML
14	Firefox	macOS	English	Enterprise	Google SSO
15	Safari	Linux	English	Free	SAML
16	Edge	Windows	Spanish	Free	Email/password

That is 16 tests covering all pairs instead of 324 exhaustive combinations -- a 95% reduction in test count.

Validating Pairwise Coverage

After generation, verify that every pair actually appears:

from itertools import combinations

def verify_pairwise_coverage(test_suite: list[dict], parameters: dict) -> list[str]:
    """Check that every pair of parameter values appears in at least one test."""
    missing_pairs = []
    param_names = list(parameters.keys())

    for p1, p2 in combinations(param_names, 2):
        for v1 in parameters[p1]:
            for v2 in parameters[p2]:
                found = any(
                    test[p1] == v1 and test[p2] == v2
                    for test in test_suite
                )
                if not found:
                    missing_pairs.append(f"({p1}={v1}, {p2}={v2})")

    return missing_pairs

# Usage
parameters = {
    "browser": ["Chrome", "Firefox", "Safari", "Edge"],
    "os": ["Windows", "macOS", "Linux"],
    "language": ["English", "Spanish", "Japanese"],
    "account": ["Free", "Pro", "Enterprise"],
    "auth": ["Email/password", "Google SSO", "SAML"],
}

missing = verify_pairwise_coverage(test_suite, parameters)
if missing:
    print(f"INCOMPLETE: {len(missing)} pairs not covered:")
    for pair in missing:
        print(f"  - {pair}")
else:
    print("All pairs covered!")