Agentic Testing Architectures
From Scripts to Agents: A Fundamental Shift
Traditional test automation is imperative: you write step-by-step scripts that execute deterministically. Agentic testing is fundamentally different -- agents observe, reason, act, and evaluate in a loop that adapts to unexpected situations. This chapter covers the architectural patterns, guardrails, real-world case studies, and economics that make agentic testing work at scale.
Chapter Contents
1. The ReAct Pattern — 01-react-pattern/
- Core Loop — The four-phase Observe-Think-Act-Evaluate loop, complete implementation, capabilities vs scripts
- Vibe-Check Mapping — Mapping ReAct phases to concrete vibe-check CLI commands, complete test walkthrough
- Practical Implementation — Error handling, prompt optimization, history management, testing the test agent
2. Multi-Agent Systems — 02-multi-agent-systems/
- Orchestrator Pattern — Centralized coordination, specialist agent design, communication protocol
- Swarm Pattern — Decentralized parallel generation, the Reconciler, partitioning strategies
- Critic-Actor Pattern — Adversarial quality improvement, review prompts, quality benchmarks
3. Case Studies — 03-case-studies/
- OpenObserve Council — Eight specialized agents, 85% flaky test reduction, shared memory design, lessons learned
4. Guardrails and Constraints — 04-guardrails-and-constraints/
- Five Guardrails — Max steps, allowed domains, timeout, token budget, action allowlist
- Production Harness — Complete ConstrainedTestHarness implementation, observability, environment profiles
- Dead Man's Switch — CI-level timeouts, process cleanup, heartbeat monitoring, the complete safety stack
5. Determinism and Cost — 05-determinism-and-cost/
- Determinism Spectrum — Four levels from recorded tests to exploratory agents, techniques for CI determinism
- Communication Protocols — Message protocol, correlation IDs, error propagation strategies, monitoring
- Cost Analysis — Token economics, four optimization strategies, ROI calculation, the interview talking point
Reading Order
- Start with Core Loop for the foundational pattern
- Then Orchestrator and Critic-Actor for multi-agent thinking
- Then OpenObserve Council for real-world validation
- Then Five Guardrails for safety design
- Finally Cost Analysis for the business case and interview talking points