How to write a release post-mortem that can serve as public evidence
A release post-mortem (a written review of an incident: timeline, cause, impact and follow-up actions) serves as primary evidence when it names the organisation, the date, the change, the time of the change, the time of detection, the impact and the tests involved. Of 163 accepted primary and secondary reports on the site's published incident pages, 26 state both the change time and the detection time. The site's verifier rejected 157 candidate reports, and 126 of those rejections were for not matching the incident definition.
What a post-mortem needs to count as primary evidence
The playbook grades a post-mortem as primary when the affected organisation publishes it about a specific dated incident.
A status page (a public page where a service posts incident updates) record counts as primary when it carries a root-cause or "what happened" update.
A write-up that describes a failure class without a specific organisation and date is graded practitioner and never counts toward the two-source rule.
Undated pages are rejected.
Two pages from the same organisation about the same incident count as one source.
- Put the publication date and the incident date at the head of the post-mortem.
- Name the organisation that owns the affected service.
- Name the change: the deploy, migration, configuration edit, upgrade or rotation that started the incident.
- State the time of the change and the time of detection in one time zone.
- State who or what detected the incident: an alert, an automated test, a customer report or a support ticket.
- State the impact in terms of service down, data affected, money affected, a named regulatory consequence, or slower or partial service.
- Name each test class that ran before the release and each one that was missing.
Record the change time and the detection time
The risk model records time to detection (minutes between the change that caused an incident and the moment it was noticed) only when a report states both the change time and the detection time.
Group pages print the median time to detection across the group's incidents with the count behind it.
An incident with fewer than 2 time-to-detection values prints nothing for that figure.
On the site's published incident pages, 26 of 163 accepted primary and secondary reports state both times.
State the impact class
The collector records one impact class per report: availability, data, money, compliance or degraded.
A report with no stated impact is recorded as unstated and is ignored by the impact grade.
On the site's published incident pages, 4 of 163 accepted primary and secondary reports state no impact.
When fewer than 2 reports on an incident state an impact, the incident's impact grade is unrated.
Name the tests that ran and the tests that were missing
A caught-by entry on an incident page carries the cited label only when an accepted report names the test class.
Without such a report, the entry carries the by design analysis label, which marks the link between cause and test class as the author's reasoning.
On the site's published incident pages, 20 of 163 accepted primary and secondary reports name a test class.
A post-mortem that names the test class and says why it passed turns a by-design entry into a cited entry on the next refresh.
Worked example
A status page post-mortem from 2023 states that an authorization service change graduated to production at 6:37 PM UTC and that alerts triggered at 6:44 PM UTC [1].
The same 2023 report names the canary ring and states that it did not send enough traffic to expose the memory leak [1].
That report therefore supplies a 7-minute time to detection and a cited canary entry on the memory leak page [1].
A post-mortem from 2023 states that alarms fired within three minutes of a foreign key migration reaching production [2].
The same 2023 post-mortem states that pre-production testing was not an adequate substitute for running the migration on the production dataset under full traffic load [2].
That statement backs a cited migration tests (migrations run against a production-shaped copy with timing and rollback checks) entry on the table lock page [2].
A post-mortem from 2025 gives the change time as 11:05 UTC and the first automated detection as 11:31 UTC [3].
The 2025 post-mortem names no test class, so its caught-by entries on the configuration page carry the by design analysis label [3].
A status page report from 2024 states that customer reports led to an incident being logged about three days after a single sign-on change [4].
The 2024 report supplies the longest time to detection among these five reports, at 4285 minutes [4].
A status page report from 2024 states that missing alerts on 4xx errors delayed identification of failed payment transactions [5].
The 2024 payment report gives no change time or detection time, so the site records no time to detection for it [5].
Why candidate reports were rejected
The site's verifier rejected 157 candidate reports across all incident types.
126 rejections were for not matching the incident definition, for example a report whose failure mechanism differed from the incident type it was collected for.
11 rejections were for not being caused by a release or change, because the report named no deploy, edit or upgrade as the start of the incident.
11 rejections were repository issues where no maintainer stated the cause.
2 rejections were for naming no specific organisation and date.
3 rejections were secondary entries that linked no primary source that still loads.
1 rejection was a second page from the same organisation about the same incident.
2 rejections were pages that no longer showed the quoted fact on load.
1 rejection was an undated page.
- Name the change as the start of the incident in the first paragraph of the post-mortem.
- Describe the failure mechanism in one sentence so a reader can match it to one incident type.
- Keep the post-mortem at a stable URL and do not edit out the timeline after publication.
- Have a maintainer state the root cause on a public repository issue before closing it.
Public reports cited
- [1]GitHub, 2023-11-03. A performance and resilience optimisation to GitHub's authorization microservice contained a memory leak exposed under high traffic; pods crashed repeatedly, authorization checks failed and pages returned 404s until the change was rolled back. primary Incident with Git Operations, Issues, Pull Requests, Actions, API Requests, Codespaces, Packages, Pages and Webhooks
- [2]Pulumi, 2023-10-18. Pulumi shipped a database migration modifying foreign keys on a table; the resulting table copy held a lock long enough to pile up queries and exhaust database connections, and Pulumi Cloud failed 74.7% of requests for about 24 minutes. primary Details of the Pulumi Outage on October 6, 2023
- [3]Cloudflare, 2025-11-18. A database permissions change was rolled out gradually across a ClickHouse cluster, so the Bot Management feature file was generated correctly on some nodes and wrongly on updated nodes, and the network alternated between good and bad files until every node produced the oversized file. primary Cloudflare outage on November 18, 2025
- [4]Phrase, 2024-02-06. A change to the SAML SSO init process moved a customer parameter from GET to POST and switched the host, so the callback redirect_uri no longer matched and customers could not log in via Strings SAML for about three days. primary Unable to log in via Strings SSO login (redirect issue)
- [5]Chargebee, 2024-08-16. Chargebee reported that after it moved its Adyen integration from the Classic to the Checkout integration on 15 August 2024, customer transactions through Adyen that used the temporary (CSE) token workflow failed with 4xx errors until the change was reverted. primary Payment Processing Issues With Adyen Gateway
Questions
How do you write a post-mortem for a release incident?
Name the organisation, the date, the change, the change time, the detection time, the impact and the test classes involved. Of 163 accepted primary and secondary reports on the site's published incident pages, 26 state both the change time and the detection time.
How do you check that a post-mortem root cause is deep enough?
Check that the root cause names the pipeline stage where the fault was introduced: code, build, migration, config, infra, third-party or release process. Each root cause on an incident page carries one of these stages.
Why did tests not catch the bug?
A post-mortem answers this when it names the test class and states why that class passed. A 2023 status page report states that its canary ring did not send enough traffic to expose a memory leak [1].
Why are some published post-mortems not used as evidence?
The site's verifier rejected 157 candidate reports. 126 of them did not match the incident definition they were collected for, and 11 named no release or change as the cause.