Modern QA2026Error Recovery in the Harness — tiles
Log inJoin
71 / 108 · 03 Agentic Testing Architectures · Production Test Harness Implementation← prev⊞ allnext →☰ Read as one page

9.6Error Recovery in the Harness

The harness should handle agent failures gracefully:

def execute_with_recovery(self, test_objective: str) -> TestResult:
    """Execute with automatic recovery from transient errors."""
    max_retries = 2
    for attempt in range(max_retries + 1):
        try:
            return self.execute(test_objective)
        except BrowserCrashError:
            if attempt < max_retries:
                self.logger.warning(f"Browser crashed, retrying (attempt {attempt + 1})")
                self.agent.reset_browser()
                self.step_count = 0
                self.token_count = 0
            else:
                return TestResult(
                    status="ERROR",
                    reason="Browser crashed after max retries"
                )
        except LLMTimeoutError:
            if attempt < max_retries:
                self.logger.warning(f"LLM timeout, retrying (attempt {attempt + 1})")
            else:
                return TestResult(
                    status="ERROR",
                    reason="LLM timeout after max retries"
                )