13.4Cost Monitoring Dashboard

Track these metrics to control spending:

class CostDashboard:
    def daily_report(self) -> dict:
        return {
            "total_tokens_today": self.get_daily_tokens(),
            "total_cost_today": self.get_daily_cost(),
            "cost_per_test": self.get_daily_cost() / self.get_daily_test_count(),
            "cache_hit_rate": self.cache.hit_rate,
            "model_distribution": {
                "haiku": self.count_by_model("haiku"),
                "sonnet": self.count_by_model("sonnet"),
                "opus": self.count_by_model("opus"),
            },
            "early_exits": self.count_early_exits(),
            "budget_utilization": self.get_daily_tokens() / self.daily_budget,
        }

Alerting Thresholds

Metric	Warning	Critical
Daily cost	> $50	> $100
Cost per test	> $0.50	> $1.00
Cache hit rate	< 15%	< 5%
Early exit rate	> 30%	> 50%