Case guides · Under pressure · UP-02
How do I run a blameless postmortem when someone really did break prod?
Blameless doesn't mean nobody did anything: the engineer gives an honest account and is not punished for it. Ask why it made sense, not why they did it. Aim the fixes at the system, protect how the person is treated, and answer a senior leader's question separately and early.
The situation
Last night's outage has a name attached. A mid-level engineer ran a migration against the production database instead of staging. Everyone on the team knows it was them; half the company has seen the incident channel. The postmortem meeting is on Thursday. The engineer has barely spoken since, a senior leader has asked you privately "what are we doing about this person?", and some of the team are quietly waiting to see whether "blameless" was ever real.
EM Field Guide is practice. It is not legal, HR or medical advice. If a situation involves dismissal, a performance plan, discrimination, safety or health, talk to your HR team or a qualified professional.
What matters most
- Blameless doesn't mean nobody did anything. John Allspaw's "Blameless PostMortems and a Just Culture" (Etsy's engineering blog, 2012) sets out the deal: the engineer gives a full, honest account of what they did, what they saw and why it made sense at the time, and in return they're not punished for it. If you break that deal once, you'll never get an honest account again.
- Ask why it made sense, not why they did it. Sidney Dekker's The Field Guide to Understanding 'Human Error' calls this local rationality: people act on what they know and the pressure they're under at that moment. Hindsight makes the right choice look obvious. The useful questions are "What did the screen show you?" and "What made staging and production look the same?"
- The system let it happen. Google's SRE book (chapter 15, "Postmortem Culture") assumes everyone acted with good intentions on the information they had, and aims the fixes at the process and tooling. If one tired person with normal access could do this, the action items are about access, guard rails and naming, not about that person.
- Blameless is not faceless. You don't have to pretend it's anonymous when everyone knows. What you protect is how the person is treated: the write-up describes actions ("the migration was run against production"), and the person is thanked, publicly, for their account.
- Handle the leader's question separately and early. A senior person asking for consequences needs an answer before Thursday, in private, or they'll ask it in the meeting. Blameless culture is decided in moments like that one.
How this lands in teams across Asia
Blameless postmortems were written in cultures where naming a mistake publicly is uncomfortable but survivable. In many teams across the region, being named, even kindly, can feel like public humiliation, and the pressure to take personal responsibility runs the other way.
- The person may want to take the blame. An engineer from a culture where accepting fault is the honourable thing may open the postmortem with a formal apology, or offer to resign. Don't let that become the story. Accept the apology privately, before the meeting: "Thank you. I accept it. In the meeting, I need your knowledge, not your apology."
- Silence protects face, and hides the causes. Colleagues may not add their own contributing mistakes ("I also approved that runbook") in front of the person or their seniors. Collect contributions in writing, one to one, before the meeting, and put them in the doc as facts about the system.
- Leadership may expect a visible consequence. In more hierarchical organisations, a senior leader may see an unpunished outage as weakness. Your job is to translate: the consequence is the fix list, with owners and dates, and a named leader (you) accountable for it. You take the accountability upward; the engineer doesn't.
- Multi-country teams read the write-up differently. A sentence that reads neutral to one office can read as a public accusation in another. Before publishing, ask someone from the affected person's office to read the draft for tone. Keep the name out of the summary that goes beyond the team.
- Seniority in the room. If the most senior person speaks first, everyone else aligns with them. Have the facilitator (not the most senior person) run the meeting, and ask the most junior people for their view first.
A way to open
“First: are you OK?”
The rest of the script is in the app.
Rehearse it
Paste this into any AI assistant:
Play the other side of a hard conversation with me. The situation: A blameless postmortem when someone really did break prod. Blameless has to hold even when the cause has a name. You are the person across the table and I am their manager. Invent a believable person and keep what is really on your mind to yourself until I ask well. After eight exchanges, tell me what I missed.
Sources
- John Allspaw, "Blameless PostMortems and a Just Culture", Etsy Code as Craft blog (May 2012).
- Sidney Dekker, The Field Guide to Understanding 'Human Error' (3rd ed., 2014): local rationality, hindsight bias, asking "what" instead of "who".
- John Lunney & Sue Lueder, "Postmortem Culture: Learning from Failure", chapter 15 of Site Reliability Engineering (2016), free at sre.google.
- Amy Edmondson, The Fearless Organization (2018): why people stay quiet about mistakes when speaking up feels risky.
Related guides
Published by Polanyi Works · Updated