6.5Systematic Output Verification with Eval Frameworks

Hand-written assertions catch known-bad patterns, but grounding and citation accuracy need continuous, metric-driven evaluation. As of July 2026, LLM eval frameworks are the standard tooling for this:

• Ragas -- RAG-specific metrics: faithfulness (is every claim in the answer grounded in the retrieved context?), answer relevancy, and context precision/recall.
• TruLens -- feedback functions for groundedness, context relevance, and answer relevance, with per-response tracing.
• OpenAI Evals -- a general harness for codifying pass/fail eval suites and running them in CI.

Wire these into the same pipeline as the security tests so a drop in grounding or citation accuracy blocks the deploy:

# eval_grounding.py -- example with Ragas-style metrics
from ragas import evaluate
from ragas.metrics import faithfulness, context_precision, context_recall

def test_rag_grounding_above_threshold(rag_eval_dataset):
    """Fail the build if answers drift away from the retrieved context."""
    result = evaluate(
        rag_eval_dataset,  # question, answer, contexts, ground_truth
        metrics=[faithfulness, context_precision, context_recall],
    )
    assert result["faithfulness"] >= 0.90, (
        f"Grounding regressed: faithfulness {result['faithfulness']:.2f} < 0.90"
    )
    assert result["context_precision"] >= 0.80

Grounding and citation-accuracy testing complement the security tests above: poisoning and injection tests prove the system rejects malicious context, while grounding metrics prove it actually uses the legitimate context instead of hallucinating an answer or a citation. Both belong in CI.