8.2The Five Guardrails

+--------------------------------------------------+
|              AGENT TEST HARNESS                  |
|                                               -  |
|  +---------------------------------------------+ |
|  | 1. MAX STEPS      (prevent infinite loops)  | |
|  | 2. ALLOWED DOMAINS (prevent unauthorized    | |
|  |                     resource access)        | |
|  | 3. TIMEOUT         (wall-clock limit)       | |
|  | 4. TOKEN BUDGET    (cost ceiling)           | |
|  | 5. ACTION ALLOWLIST (restrict operations)   | |
|  +---------------------------------------------+ |
|                                                  |
|  Agent operates freely WITHIN these boundaries   |
+--------------------------------------------------+

Guardrail 1: Max Steps

Prevents the agent from looping infinitely. If the agent takes more than N actions without completing its objective, the test is aborted.

max_steps: int = 30  # Default: 30 actions before forced stop

How to set the limit:

• Count the steps in your longest manual test flow (e.g., 15 steps for a checkout flow)
• Multiply by 2 to allow for the agent's exploration overhead
• That is your max_steps

What happens when exceeded:

if self.step_count >= self.config.max_steps:
    return TestResult(
        status="ABORTED",
        reason=f"Max steps ({self.config.max_steps}) exceeded. "
               f"Agent could not complete objective in {self.step_count} steps.",
        steps_taken=self.step_count,
        last_observation=self.last_observation
    )

Guardrail 2: Allowed Domains

Prevents the agent from navigating to URLs outside the test environment. Without this, an agent testing a web app could follow links to external sites, trigger OAuth flows, or even interact with production systems.

allowed_domains: list[str] = ["staging.myapp.com", "api.staging.myapp.com"]

Implementation:

def is_domain_allowed(self, url: str) -> bool:
    from urllib.parse import urlparse
    domain = urlparse(url).netloc
    if not self.config.allowed_domains:
        return True  # No restriction configured
    return any(
        domain == allowed or domain.endswith(f".{allowed}")
        for allowed in self.config.allowed_domains
    )

Real-world example: An agent testing a login page follows the "Forgot Password" link, which redirects to an email provider's OAuth page. Without domain allowlisting, the agent would interact with Google's login page. With it, the action is blocked and logged.

Guardrail 3: Timeout

A wall-clock limit that prevents runaway test execution. Even if the agent is within its step and token budgets, a slow-responding server or a browser hang could keep the test running indefinitely.

timeout_seconds: int = 300  # 5-minute wall clock limit

Implementation:

import time

def check_timeout(self) -> str | None:
    elapsed = time.time() - self.start_time
    if elapsed > self.config.timeout_seconds:
        return (f"Timeout ({self.config.timeout_seconds}s) exceeded. "
                f"Elapsed: {elapsed:.1f}s")
    return None

Pro tip: Set two timeout layers:

• Per-action timeout: 30 seconds per browser action (catches hangs)
• Per-test timeout: 300 seconds for the entire test (catches slow loops)

Guardrail 4: Token Budget

A cost ceiling that prevents the agent from making unlimited LLM calls. Each call costs tokens, and costs accumulate fast in multi-step agents.

max_tokens: int = 50_000  # Token budget per test run

Why this matters financially:

Agent behavior	Tokens per test	Cost (Claude Sonnet)
10-step simple test	~10K	$0.05
30-step complex test	~30K	$0.15
Runaway agent (no budget)	~500K+	$2.50+

Implementation:

def track_tokens(self, response) -> None:
    self.token_count += response.usage.input_tokens + response.usage.output_tokens
    if self.token_count >= self.config.max_tokens:
        raise TokenBudgetExceeded(
            f"Token budget ({self.config.max_tokens}) exhausted. "
            f"Used: {self.token_count}"
        )

Guardrail 5: Action Allowlist

Restricts which actions the agent can perform. In production monitoring, you want read-only actions. In CI, you want everything except destructive operations.

allowed_actions: list[str] = ["NAVIGATE", "CLICK", "TYPE", "ASSERT", "SCREENSHOT"]
# Blocked: "DELETE", "DROP", "RESET", "ADMIN_*"

Implementation:

def is_action_allowed(self, action) -> bool:
    if self.config.allowed_actions:
        if action.type not in self.config.allowed_actions:
            return False
    # Additional checks for specific action types
    if action.type == "NAVIGATE":
        if not self.is_domain_allowed(action.url):
            return False
    if action.type == "TYPE" and action.selector.contains("password"):
        # Never type real credentials -- use test tokens
        if not action.text.startswith("test_"):
            return False
    return True