Dashboards and Release Decisions
Quality engineers communicate through data. Two of the highest-leverage activities at this level are building dashboards that drive action (not just display numbers) and establishing release criteria that remove politics from go/no-go decisions.
Dashboards That Drive Decisions
Anti-Pattern: One dashboard for all audiences. Developers, QA, product managers, and leadership all see the same view — so none of them find what they need, and the dashboard is ignored.
Pattern: Four dashboards for four audiences, each answering a different question.
| Dashboard | Audience | Core Question | Key Metrics |
|---|---|---|---|
| Developer | Individual developers | "What did I break?" | Failed tests per PR, flaky tests in my area, build time |
| QA | QA team | "Is testing effective?" | Flaky rate trend, coverage gaps, escaped defects, test suite health |
| Product | Product managers | "Is this release ready?" | Feature test status, regression results, open blockers, risk areas |
| Leadership | Engineering leadership | "Is quality improving?" | Escaped defect trend, incident rate, MTTR, deployment frequency |
A dashboard that nobody checks is a dashboard that does not exist. Design for the audience's actual workflow — embed results in PRs for developers, send weekly summaries for leadership, integrate with release checklists for product.
When QA Should Say "No"
Anti-Pattern: Release decisions are made based on pressure, politics, or "it's probably fine." QA raises concerns verbally, gets overridden, and then absorbs blame when the release breaks.
Pattern: Pre-agreed release blocking criteria that remove politics from the decision. The criteria exist before the release conversation happens.
Release Blocking Criteria (Example)
- Any P0/P1 bug in the release scope
- Test suite pass rate below 95% (excluding known flaky tests)
- Untested critical paths (identified in risk matrix)
- Performance regression above threshold
- Security vulnerability in changed code
The Escalation Framework
When you believe a release should be blocked:
- Document — Write down what you found, with evidence (screenshots, logs, test results)
- Quantify — Express the risk in business terms: "This payment bug affects approximately X% of transactions"
- Present options — "We can delay 2 days for a fix, release with a feature flag, or release and accept the risk"
- Accept the decision — If leadership decides to release despite your recommendation, that is their prerogative. Your job was to make the risk visible and quantified. Document the decision.
The goal is not to block releases. The goal is to make risk-informed decisions. Sometimes the business reason to release outweighs the quality risk — and that is a legitimate decision when made with full information.
Key Takeaways
- Build separate dashboards for developers, QA, product, and leadership — one dashboard for all audiences serves none of them
- Embed quality data into existing workflows (PR comments, release checklists) rather than requiring people to visit a dashboard
- Establish release blocking criteria before the release conversation, not during it
- Escalate with data: document the issue, quantify the risk in business terms, present options
- Accept that sometimes business context outweighs quality risk — your job is to make the risk visible, not to have veto power