The Five Guardrails for Agentic Testing

Why Guardrails Are Non-Negotiable

Unconstrained agents are dangerous. They can run indefinitely, access resources they should not, produce non-deterministic results, and consume unbounded tokens. A well-designed test harness constrains the agent while preserving its ability to adapt.

The principle: freedom within boundaries. The agent operates freely inside the guardrails, but the guardrails are hard limits that cannot be exceeded.

The Five Guardrails

+--------------------------------------------------+
|              AGENT TEST HARNESS                  |
|                                               -  |
|  +---------------------------------------------+ |
|  | 1. MAX STEPS      (prevent infinite loops)  | |
|  | 2. ALLOWED DOMAINS (prevent unauthorized    | |
|  |                     resource access)        | |
|  | 3. TIMEOUT         (wall-clock limit)       | |
|  | 4. TOKEN BUDGET    (cost ceiling)           | |
|  | 5. ACTION ALLOWLIST (restrict operations)   | |
|  +---------------------------------------------+ |
|                                                  |
|  Agent operates freely WITHIN these boundaries   |
+--------------------------------------------------+

Guardrail 1: Max Steps

Prevents the agent from looping infinitely. If the agent takes more than N actions without completing its objective, the test is aborted.

max_steps: int = 30  # Default: 30 actions before forced stop

How to set the limit:

• Count the steps in your longest manual test flow (e.g., 15 steps for a checkout flow)
• Multiply by 2 to allow for the agent's exploration overhead
• That is your max_steps

What happens when exceeded:

if self.step_count >= self.config.max_steps:
    return TestResult(
        status="ABORTED",
        reason=f"Max steps ({self.config.max_steps}) exceeded. "
               f"Agent could not complete objective in {self.step_count} steps.",
        steps_taken=self.step_count,
        last_observation=self.last_observation
    )

Guardrail 2: Allowed Domains

Prevents the agent from navigating to URLs outside the test environment. Without this, an agent testing a web app could follow links to external sites, trigger OAuth flows, or even interact with production systems.

allowed_domains: list[str] = ["staging.myapp.com", "api.staging.myapp.com"]

Implementation:

def is_domain_allowed(self, url: str) -> bool:
    from urllib.parse import urlparse
    domain = urlparse(url).netloc
    if not self.config.allowed_domains:
        return True  # No restriction configured
    return any(
        domain == allowed or domain.endswith(f".{allowed}")
        for allowed in self.config.allowed_domains
    )

Real-world example: An agent testing a login page follows the "Forgot Password" link, which redirects to an email provider's OAuth page. Without domain allowlisting, the agent would interact with Google's login page. With it, the action is blocked and logged.

Guardrail 3: Timeout

A wall-clock limit that prevents runaway test execution. Even if the agent is within its step and token budgets, a slow-responding server or a browser hang could keep the test running indefinitely.

timeout_seconds: int = 300  # 5-minute wall clock limit

Implementation:

import time

def check_timeout(self) -> str | None:
    elapsed = time.time() - self.start_time
    if elapsed > self.config.timeout_seconds:
        return (f"Timeout ({self.config.timeout_seconds}s) exceeded. "
                f"Elapsed: {elapsed:.1f}s")
    return None

Pro tip: Set two timeout layers:

• Per-action timeout: 30 seconds per browser action (catches hangs)
• Per-test timeout: 300 seconds for the entire test (catches slow loops)

Guardrail 4: Token Budget

A cost ceiling that prevents the agent from making unlimited LLM calls. Each call costs tokens, and costs accumulate fast in multi-step agents.

max_tokens: int = 50_000  # Token budget per test run

Why this matters financially:

Implementation:

def track_tokens(self, response) -> None:
    self.token_count += response.usage.input_tokens + response.usage.output_tokens
    if self.token_count >= self.config.max_tokens:
        raise TokenBudgetExceeded(
            f"Token budget ({self.config.max_tokens}) exhausted. "
            f"Used: {self.token_count}"
        )

Guardrail 5: Action Allowlist

Restricts which actions the agent can perform. In production monitoring, you want read-only actions. In CI, you want everything except destructive operations.

allowed_actions: list[str] = ["NAVIGATE", "CLICK", "TYPE", "ASSERT", "SCREENSHOT"]
# Blocked: "DELETE", "DROP", "RESET", "ADMIN_*"

Implementation:

def is_action_allowed(self, action) -> bool:
    if self.config.allowed_actions:
        if action.type not in self.config.allowed_actions:
            return False
    # Additional checks for specific action types
    if action.type == "NAVIGATE":
        if not self.is_domain_allowed(action.url):
            return False
    if action.type == "TYPE" and action.selector.contains("password"):
        # Never type real credentials -- use test tokens
        if not action.text.startswith("test_"):
            return False
    return True

Guardrail Configuration by Environment

Different environments need different guardrail profiles:

Loading Configuration from Environment

from dataclasses import dataclass, field
import os

@dataclass
class HarnessConfig:
    max_steps: int = field(
        default_factory=lambda: int(os.getenv("AGENT_MAX_STEPS", "30"))
    )
    timeout_seconds: int = field(
        default_factory=lambda: int(os.getenv("AGENT_TIMEOUT", "300"))
    )
    max_tokens: int = field(
        default_factory=lambda: int(os.getenv("AGENT_MAX_TOKENS", "50000"))
    )
    allowed_domains: list[str] = field(
        default_factory=lambda: os.getenv("AGENT_ALLOWED_DOMAINS", "").split(",")
    )
    allowed_actions: list[str] = field(
        default_factory=lambda: os.getenv(
            "AGENT_ALLOWED_ACTIONS",
            "NAVIGATE,CLICK,TYPE,ASSERT,SCREENSHOT"
        ).split(",")
    )

Guardrail Violations: Logging and Alerting

Every guardrail violation should be logged with enough context to diagnose the issue:

@dataclass
class GuardrailViolation:
    guardrail: str          # "max_steps", "timeout", "token_budget", etc.
    threshold: str          # "30 steps", "300s", "50000 tokens"
    actual: str             # "31 steps", "305s", "51234 tokens"
    test_objective: str
    last_observation: str   # What the agent saw before violation
    last_action: str        # What the agent tried to do
    history_summary: str    # Compressed history for debugging
    screenshot_path: str | None

Key Takeaway

The five guardrails (Max Steps, Allowed Domains, Timeout, Token Budget, Action Allowlist) form a defense-in-depth system. Each catches a different failure mode. Together, they make agentic testing safe for CI/CD pipelines where an unconstrained agent could block deployments, consume expensive tokens, or interact with systems outside the test scope.