Release incident types: public reports, risk grades and tests
The playbook publishes 58 release incident types in 10 groups, backed by 163 accepted public reports.
An incident type is published only when at least two organisations have a dated public report of it, one within the last 5 years.
Incident types by group
Payments and billing
| Checkout blocked for segment | Likelihood: low (2 public reports in 5 years) | Impact: high (2 of 2 reports: money) | 2 |
| Double charge after deploy | Likelihood: not published (1 public report in 5 years) | Impact: high (2 of 2 reports: money) | 2 |
| Subscription renewal skipped | Likelihood: low (2 public reports in 5 years) | Impact: high (2 of 2 reports: money) | 2 |
| Tax calculation regression | Likelihood: low (2 public reports in 5 years) | Impact: high (2 of 2 reports: money and compliance) | 2 |
Data and migrations
| Encoding or timezone shift | Likelihood: not published (1 public report in 5 years) | Impact: high (2 of 2 reports: data) | 2 |
| Irreversible migration fails midway | Likelihood: low (2 public reports in 5 years) | Impact: medium (2 of 4 reports: availability) | 4 |
| Long-running migration locks table | Likelihood: low (2 public reports in 5 years) | Impact: medium (3 of 3 reports: availability) | 3 |
| Missing index after migration | Likelihood: medium (3 public reports in 5 years) | Impact: medium (4 of 5 reports: availability) | 5 |
| ORM upgrade changes query semantics | Likelihood: low (2 public reports in 5 years) | Impact: high (3 of 3 reports: data) | 3 |
| Schema drift between services | Likelihood: medium (4 public reports in 5 years) | Impact: medium (2 of 4 reports: availability) | 4 |
| Soft-delete flag ignored | Likelihood: low (2 public reports in 5 years) | Impact: high (1 of 2 reports: data) | 2 |
Authentication and access
| All sessions invalidated | Likelihood: medium (3 public reports in 5 years) | Impact: medium (3 of 3 reports: availability) | 3 |
| API key rotation breaks integrations | Likelihood: not published (1 public report in 5 years) | Impact: medium (2 of 2 reports: availability) | 2 |
| CORS policy blocks frontend | Likelihood: medium (3 public reports in 5 years) | Impact: medium (3 of 3 reports: availability) | 3 |
| Password reset link broken | Likelihood: low (2 public reports in 5 years) | Impact: medium (3 of 3 reports: availability) | 3 |
| Permission escalation via role change | Likelihood: medium (3 public reports in 5 years) | Impact: high (4 of 4 reports: data) | 4 |
| SSO login broken | Likelihood: medium (5 public reports in 5 years) | Impact: medium (5 of 5 reports: availability) | 5 |
Performance and capacity
| Background job queue backlog | Likelihood: medium (3 public reports in 5 years) | Impact: low (2 of 3 reports: degraded) | 3 |
| Cache invalidation storm | Likelihood: low (2 public reports in 5 years) | Impact: medium (2 of 2 reports: availability) | 2 |
| Cold start latency regression | Likelihood: low (2 public reports in 5 years) | Impact: low (2 of 2 reports: degraded) | 2 |
| Connection pool exhausted | Likelihood: medium (3 public reports in 5 years) | Impact: medium (3 of 4 reports: availability) | 5 |
| Memory leak in new release | Likelihood: low (2 public reports in 5 years) | Impact: medium (2 of 2 reports: availability) | 3 |
| Thundering herd on restart | Likelihood: low (2 public reports in 5 years) | Impact: medium (2 of 2 reports: availability) | 2 |
Configuration and feature flags
| Config change without restart semantics | Likelihood: low (2 public reports in 5 years) | Impact: medium (2 of 2 reports: availability) | 2 |
| Environment mismatch: production points to staging | Likelihood: low (2 public reports in 5 years) | Impact: unrated (fewer than 2 reports state impact) | 2 |
| Flag default flips on missing config | Likelihood: low (2 public reports in 5 years) | Impact: low (2 of 2 reports: degraded) | 2 |
| Flag enabled for wrong segment | Likelihood: low (2 public reports in 5 years) | Impact: medium (1 of 2 reports: availability) | 3 |
| Rate limit misconfigured | Likelihood: medium (4 public reports in 5 years) | Impact: medium (4 of 4 reports: availability) | 4 |
Mobile release
| API version dropped while old app live | Likelihood: not published (1 public report in 5 years) | Impact: medium (2 of 2 reports: availability) | 2 |
| App store rollback not possible | Likelihood: medium (5 public reports in 5 years) | Impact: medium (2 of 5 reports: availability) | 5 |
| Crash on older OS version | Likelihood: low (2 public reports in 5 years) | Impact: medium (2 of 2 reports: availability) | 2 |
| Deep link broken after routing change | Likelihood: low (2 public reports in 5 years) | Impact: medium (2 of 2 reports: availability) | 2 |
| In-app purchase receipt validation fails | Likelihood: low (2 public reports in 5 years) | Impact: high (2 of 2 reports: money) | 2 |
| Push notification token reset | Likelihood: low (2 public reports in 5 years) | Impact: medium (2 of 2 reports: availability) | 2 |
Third-party integrations
| Certificate expired | Likelihood: medium (4 public reports in 5 years) | Impact: medium (4 of 4 reports: availability) | 4 |
| Dependency removed from registry | Likelihood: not published (1 public report in 5 years) | Impact: medium (3 of 3 reports: availability) | 3 |
| Email or SMS provider template broken | Likelihood: not published (1 public report in 5 years) | Impact: medium (1 of 2 reports: availability) | 2 |
| OAuth callback URL not whitelisted | Likelihood: low (2 public reports in 5 years) | Impact: medium (3 of 3 reports: availability) | 3 |
| Provider API change breaks contract | Likelihood: medium (4 public reports in 5 years) | Impact: medium (2 of 4 reports: availability) | 4 |
| SDK upgrade changes behaviour | Likelihood: medium (3 public reports in 5 years) | Impact: medium (2 of 3 reports: availability) | 3 |
| Third-party rate limit hit | Likelihood: low (2 public reports in 5 years) | Impact: medium (2 of 2 reports: availability) | 2 |
Frontend web
| Analytics event schema broken | Likelihood: low (2 public reports in 5 years) | Impact: high (2 of 2 reports: data) | 2 |
| Asset cache serves old bundle | Likelihood: medium (3 public reports in 5 years) | Impact: medium (2 of 3 reports: availability) | 3 |
| Critical path broken in one browser | Likelihood: low (2 public reports in 5 years) | Impact: medium (2 of 2 reports: availability) | 2 |
| CSP blocks required script | Likelihood: low (2 public reports in 5 years) | Impact: medium (2 of 2 reports: availability) | 2 |
| Form submission silently fails | Likelihood: not published (1 public report in 5 years) | Impact: high (2 of 2 reports: data) | 2 |
| Localisation strings missing | Likelihood: low (2 public reports in 5 years) | Impact: low (2 of 2 reports: degraded) | 2 |
Infrastructure and deploy
| Autoscaling policy misconfigured | Likelihood: medium (4 public reports in 5 years) | Impact: medium (5 of 5 reports: availability) | 5 |
| Container image base change | Likelihood: low (2 public reports in 5 years) | Impact: medium (3 of 3 reports: availability) | 3 |
| Disk full from new log volume | Likelihood: low (2 public reports in 5 years) | Impact: medium (2 of 3 reports: availability) | 3 |
| DNS change propagation outage | Likelihood: medium (3 public reports in 5 years) | Impact: medium (3 of 3 reports: availability) | 3 |
| IAM policy change blocks service | Likelihood: medium (3 public reports in 5 years) | Impact: medium (3 of 3 reports: availability) | 3 |
| Rolling deploy mixed versions | Likelihood: not published (1 public report in 5 years) | Impact: medium (1 of 2 reports: availability) | 2 |
Observability and rollback
| Alerts silent after metric rename | Likelihood: low (2 public reports in 5 years) | Impact: unrated (fewer than 2 reports state impact) | 2 |
| Error budget burned by retry storm | Likelihood: low (2 public reports in 5 years) | Impact: medium (2 of 2 reports: availability) | 2 |
| Partial rollback leaves inconsistent state | Likelihood: low (2 public reports in 5 years) | Impact: low (2 of 3 reports: degraded) | 3 |
| Release marker missing in monitoring | Likelihood: not published (1 public report in 5 years) | Impact: medium (2 of 2 reports: availability) | 3 |
| Rollback fails due to forward-only change | Likelihood: low (2 public reports in 5 years) | Impact: medium (2 of 2 reports: availability) | 4 |
Risk grid
| Likelihood \ Impact | Impact: high | Impact: medium | Impact: low |
|---|---|---|---|
| Likelihood: high | 0 | 0 | 0 |
| Likelihood: medium | 1 | 15 | 1 |
| Likelihood: low | 7 | 19 | 4 |
| Impact unrated or likelihood not graded | 11 | ||
Table equivalent: incidents per cell
Data status
| Page type | Published | Withheld |
|---|---|---|
| Incident pages | 58 | withheld 26 |
| Group pages | 10 | withheld 0 |
| Checklist pages | 58 | withheld 26 |
Method
Each incident page follows the same 13 sections: definition, symptoms, root causes, public examples, test classes, risk, two checklists, catalogue entries, a fact found only on that page, related incidents and questions. Likelihood counts public reports from the last 5 years; impact follows the impact class each report states.
Questions
What are the types of production incidents?
The playbook groups release incident types into payments and billing, data and migrations, authentication and access, performance and capacity, configuration and feature flags, mobile release, third-party integrations, frontend web, infrastructure and deploy and observability and rollback. 58 of 84 types are published.
What bugs cause production incidents after a release?
Root causes on published pages are introduced at these stages: code (33), release-process (27), config (23), third-party (16), infra (8), migration (7), build (3). The count is incident types naming the stage.
Which pipeline stages introduce release incidents most often?
The code stage is named in 33 of 58 published incident types, more than any other stage.
How many incident types and public reports does the playbook document?
58 incident types are published, backed by 163 accepted public reports. 26 incident routes are withheld.
How are the likelihood and impact labels assigned?
Likelihood counts accepted public reports from distinct organisations in the last 5 years: 6 or more is high, 3 to 5 medium, 2 low. Impact follows the majority impact class the reports state: data, money or compliance is high, availability medium, degraded low.
Why is an incident type missing from the playbook?
An incident type is withheld when fewer than two organisations have an accepted public report of it, when no report is from the last 5 years, or when its page content fails validation. /methodology/not-published/ lists each withheld route with its reason.