Test classes and the release incidents each one catches
The playbook uses 12 test classes, from unit tests before merge to synthetic monitoring after deploy. Each incident page names one to three of them, labelled as cited in a public report or as by design analysis.
All test classes
| Class | Stage | Incidents | Cited | Tools | Providers |
|---|---|---|---|---|---|
| Unit tests tests of one function or class in isolation | pre-merge | 9 | 1 | 9 | 4 |
| Contract tests consumer and provider agree on request and response shape before deploy | pre-merge, pre-deploy | 9 | 0 | 3 | 4 |
| Integration tests real dependencies (database, queue, provider sandbox) exercised together | pre-deploy | 21 | 4 | 5 | 22 |
| End-to-end tests user flows through the real UI against a deployed environment | pre-deploy, post-deploy | 17 | 1 | 32 | 51 |
| Migration tests migrations run against a production-shaped copy with timing and rollback checks | pre-deploy | 4 | 3 | 8 | 1 |
| Load and performance tests traffic replay or synthetic load with latency and resource budgets | pre-deploy | 14 | 3 | 10 | 47 |
| Security tests automated and manual checks for auth, authorisation, secrets and injection | pre-deploy | 3 | 1 | 11 | 37 |
| Configuration and environment tests checks that config, flags, secrets and endpoints resolve correctly per environment | pre-deploy, post-deploy | 24 | 1 | 5 | 0 |
| Compatibility tests the same flow across browsers, OS versions and device classes | pre-deploy | 2 | 0 | 20 | 31 |
| Canary and progressive delivery checks a small share of traffic runs the new version with automated metric comparison | post-deploy | 26 | 1 | 7 | 0 |
| Synthetic monitoring scripted checks of critical flows against production on a schedule | post-deploy | 20 | 2 | 6 | 0 |
| Chaos and resilience tests injected failures (dependency down, latency, restart) to check degradation and recovery | pre-deploy, post-deploy | 9 | 0 | 11 | 2 |
Questions
What is test classes?
The site uses 12 test classes, each defined by what it catches and what it cannot catch.
Which incidents is test classes used for?
58 published incident types each list one to three test classes.