3.7Post-Mortems: When You Didn't Block and Should Have

Every QA engineer has this experience: you had doubts about a release, you did not push hard enough, and the release caused a production incident. The post-mortem is where you turn this failure into a systemic improvement.

The Blameless Post-Mortem

A blameless post-mortem focuses on the system that allowed the failure, not the individuals who made decisions:

Question	Purpose
What happened?	Establish the timeline and facts
What was the impact?	Quantify the damage
Why did our process not catch this?	Identify systemic gaps
What would have caught this?	Define the missing safeguard
What will we change?	Commit to a concrete improvement

Post-Mortem Example

Incident: 2,400 orders charged without receiving confirmation email (v2.4.0, March 15-16)

Root cause: Email service integration was tested against a mock that did not replicate the production rate limit. Under production load, the email service throttled requests and 15% of emails were silently dropped.

Why QA did not catch it:

• Staging environment email mock has no rate limiting
• Load testing did not include the email notification step
• QA raised a concern about email testing but did not have data to quantify the risk

Action items:

1. Add rate limiting to the staging email mock (DevOps, Sprint 26)
2. Include notification flows in load test scenarios (QA, Sprint 26)
3. Add monitoring alert for email delivery rate dropping below 95% (DevOps, Sprint 26)
4. Update release checklist to include "notification flow tested under load" (QA, Sprint 26)