15 / 67 · 06 Observability-Driven Testing · A/B Tests as Quality Gates← prev⊞ allnext →☰ Read as one page
3.2Quality-Focused A/B Metrics
| Metric Category | Specific Metrics | What Degradation Indicates |
|---|---|---|
| Functional | Error rate, crash rate, retry rate | Bugs in the new version |
| Performance | LCP, TTFB, API latency | Performance regression |
| Engagement | Bounce rate, session duration, task completion | UX degradation |
| Business | Conversion rate, revenue per session | Feature harms business |
| Operational | CPU usage, memory, queue depth | Resource efficiency regression |
The power of using A/B testing infrastructure for quality is that you get causal evidence, not just correlation. Because users are randomly assigned to control and treatment groups, any difference in metrics is caused by the code change, not by confounding factors.