10.5Cost-Aware Load Testing

LLM load tests cost real money. Plan your budget:

# estimate_load_test_cost.py
def estimate_cost(
    requests: int,
    avg_prompt_tokens: int = 200,
    avg_completion_tokens: int = 150,
    model: str = "gpt-4o",
) -> dict:
    """Estimate the cost of a load test run."""
    # Historical published rates (2024-era models), kept for illustration.
    # Provider pricing changes often -- refresh this table with current
    # per-token rates for the models you actually run (as of July 2026:
    # GPT-5.5, Claude Opus 4.8, Gemini 3.1 Pro).
    pricing = {
        "gpt-4o": {"input": 2.50 / 1_000_000, "output": 10.00 / 1_000_000},
        "gpt-4o-mini": {"input": 0.15 / 1_000_000, "output": 0.60 / 1_000_000},
        "claude-3-5-sonnet": {"input": 3.00 / 1_000_000, "output": 15.00 / 1_000_000},
    }

    if model not in pricing:
        return {"error": f"Unknown model: {model}"}

    p = pricing[model]
    input_cost = requests * avg_prompt_tokens * p["input"]
    output_cost = requests * avg_completion_tokens * p["output"]

    return {
        "model": model,
        "requests": requests,
        "estimated_input_cost": f"${input_cost:.2f}",
        "estimated_output_cost": f"${output_cost:.2f}",
        "estimated_total_cost": f"${input_cost + output_cost:.2f}",
    }

# Example: 1000 requests priced at gpt-4o's historical rates
print(estimate_cost(1000, model="gpt-4o"))
# {'model': 'gpt-4o', 'requests': 1000,
#  'estimated_input_cost': '$0.50', 'estimated_output_cost': '$1.50',
#  'estimated_total_cost': '$2.00'}

Cost Optimization Tips for Load Testing

1. Use the cheapest model for rate limit and throughput testing. You do not need a frontier model like GPT-5.5 or Claude Opus 4.8 to test whether your rate limiter works. Use your provider's smallest tier (the mini/lite class).
2. Minimize max_tokens. Set max_tokens: 5 for tests that only measure latency, not output quality.
3. Cache where possible. If your system has a semantic cache, verify it works under load by sending repeated queries.
4. Test in short bursts. Instead of a 30-minute sustained test, use a 5-minute ramp with aggressive rate increases to find the breaking point quickly.
5. Budget per test run. Set a hard cost ceiling (e.g., $10 per CI run) and design tests within that constraint.