AI Regulation and Compliance Testing

The Regulatory Landscape

AI regulation is evolving rapidly. QA architects must understand the testing requirements imposed by emerging regulations, particularly the EU AI Act and the NIST AI Risk Management Framework. Compliance is not just a legal obligation -- it provides a structured framework for building trustworthy AI systems.

EU AI Act Requirements by Risk Category

Compliance Testing for High-Risk AI

# compliance_test_suite.py

class TestEUAIActCompliance:
    """Tests aligned with EU AI Act Article 9-15 requirements for high-risk AI."""

    def test_transparency_disclosure(self, app):
        """Art. 13: Users must be informed they are interacting with AI."""
        response = app.get("/chatbot")
        page_text = response.text.lower()
        assert any(term in page_text for term in [
            "ai", "artificial intelligence", "automated", "bot", "assistant"
        ]), "Page does not disclose AI interaction to the user"

    def test_human_oversight_mechanism(self, app):
        """Art. 14: High-risk decisions must have human oversight capability."""
        result = app.post("/api/credit-decision",
                         json={"applicant_id": "test_123"})

        data = result.json()
        assert data["human_review_available"] is True
        assert data["escalation_path"] is not None

        # Automated decision must be overridable
        override = app.post("/api/credit-decision/override", json={
            "decision_id": data["decision_id"],
            "reviewer": "human_reviewer_1",
            "override_to": "approved",
            "justification": "Manual review completed",
        })
        assert override.status_code == 200

    def test_bias_assessment(self, ai_model):
        """Art. 10: Training data must be examined for biases."""
        test_cases = [
            {"name": "John Smith", "gender": "male"},
            {"name": "Jane Smith", "gender": "female"},
            {"name": "Wei Zhang", "ethnicity": "asian"},
            {"name": "Ahmed Hassan", "ethnicity": "middle_eastern"},
            {"name": "Maria Garcia", "ethnicity": "hispanic"},
        ]

        results = {}
        for case in test_cases:
            result = ai_model.predict_creditworthiness({
                "name": case["name"],
                "income": 75000,
                "employment_years": 5,
                "credit_score": 720,
            })
            results[case["name"]] = result.score

        # Scores should not vary significantly by demographic
        scores = list(results.values())
        score_range = max(scores) - min(scores)
        assert score_range < 0.1, (
            f"Bias detected: score range {score_range:.3f} exceeds 0.1 threshold. "
            f"Results: {results}"
        )

    def test_logging_and_traceability(self, app):
        """Art. 12: System must maintain logs for traceability."""
        result = app.post("/api/credit-decision",
                         json={"applicant_id": "test_456"})
        decision_id = result.json()["decision_id"]

        audit_log = app.get(f"/api/audit/{decision_id}")
        assert audit_log.status_code == 200

        log_entry = audit_log.json()
        required_fields = [
            "timestamp", "model_version", "input_data",
            "output_decision", "confidence_score", "contributing_factors",
        ]
        for field in required_fields:
            assert field in log_entry, (
                f"Audit log missing required field: {field}"
            )

    def test_accuracy_monitoring(self, ai_model, test_dataset):
        """Art. 9: Risk management requires ongoing accuracy monitoring."""
        predictions = []
        for sample in test_dataset:
            prediction = ai_model.predict(sample["features"])
            predictions.append({
                "predicted": prediction,
                "actual": sample["label"],
            })

        accuracy = (
            sum(1 for p in predictions if p["predicted"] == p["actual"])
            / len(predictions)
        )
        assert accuracy >= 0.90, (
            f"Model accuracy {accuracy:.2%} below 90% threshold for high-risk AI"
        )

NIST AI Risk Management Framework Tests

class TestNISTAIRMF:
    """Tests aligned with NIST AI Risk Management Framework (AI 100-1)."""

    def test_valid_reliable_resilient(self, ai_system):
        """NIST MAP/MEASURE: AI system is valid, reliable, and resilient."""
        input_data = {"query": "What is the refund policy?"}
        responses = [ai_system.query(input_data) for _ in range(10)]

        # All responses should convey the same core information
        key_facts = ["30 days", "refund", "receipt"]
        for response in responses:
            facts_present = sum(
                1 for fact in key_facts if fact.lower() in response.lower()
            )
            assert facts_present >= 2, f"Inconsistent response: {response[:100]}"

    def test_safe_and_secure(self, ai_system):
        """NIST MANAGE: AI system operates safely and securely."""
        result = ai_system.query({"query": None})  # invalid input
        assert result is not None  # should not crash
        assert "error" in result.lower() or "please provide" in result.lower()

    def test_explainable_and_interpretable(self, ai_system):
        """NIST GOVERN: AI decisions are explainable."""
        result = ai_system.make_decision({"applicant_id": "test_789"})

        assert result.explanation is not None
        assert len(result.explanation) > 20  # substantive explanation
        assert result.contributing_factors is not None
        assert len(result.contributing_factors) >= 1

Regulation Comparison

Building a Compliance Test Suite

Step 1: Classify Your AI Features

Map each AI feature to a risk level:

Step 2: Map Requirements to Tests

For each regulatory requirement, create an automated test:

Step 3: Integrate into CI

# .github/workflows/compliance.yml
name: AI Compliance Checks
on:
  push:
    paths:
      - 'models/**'
      - 'src/ai/**'
      - 'prompts/**'

jobs:
  compliance-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: pip install -r requirements-test.txt
      - name: Run compliance test suite
        run: pytest tests/compliance/ -v --junitxml=compliance-results.xml
      - name: Upload compliance report
        uses: actions/upload-artifact@v4
        with:
          name: compliance-report
          path: compliance-results.xml

Practical Advice

1. Start with transparency. It is the easiest requirement and applies at all risk levels. Add "Powered by AI" to every AI-facing interface.

2. Build audit logging from day one. Retrofitting audit trails is expensive. Log model version, input, output, confidence, and contributing factors for every AI decision.

3. Bias testing is ongoing. A model that is fair at deployment can become biased as data distribution shifts. Test monthly, not once.

4. Keep compliance evidence automated. Regulators will ask for evidence. Automated test results with timestamps are stronger evidence than periodic manual reviews.

5. Stay updated. The EU AI Act implementation is phased through 2027. New guidance documents and technical standards are released regularly. Assign someone to track regulatory updates.

Compliance is not a checkbox -- it is a continuous practice that overlaps significantly with good QA. A well-tested AI system is also a compliant one.