Canary and progressive delivery checks: what they catch and the release incidents they cover
Canary and progressive delivery checks (a small share of traffic runs the new version with automated metric comparison) run at the post-deploy stage.
26 published incident types list canary and progressive delivery checks as a class that catches them; 1 of these links are cited in a public report and 25 are by design analysis.
What canary and progressive delivery checks catch and cannot catch
Catches
- regressions visible in traffic metrics
- memory growth
- error rate rise
Cannot catch
- faults in traffic the canary never receives
- batch jobs
- scheduled tasks
Both lists come from the site's test class definitions.
Incident types this class catches
| Incident | Stage | Limitation on that page | Basis |
|---|---|---|---|
| Memory leak in new release | post-deploy | The canary ring in report [1] did not expose the authorization service to enough traffic to discover the leak, so the build graduated to production at 6:37 PM UTC. | cited in [1] |
| App store rollback not possible | post-deploy | Halting a phased store rollout stops new installs of the broken version, but users who already updated keep it until a reviewed fix ships. | by design analysis |
| Autoscaling policy misconfigured | post-deploy | A canary that splits request traffic does not cover a scaling rule that applies to the whole cluster the moment it ships. | by design analysis |
| Connection pool exhausted | post-deploy | A canary on part of the pods can show no pooler saturation, as at 1, 5, 20 and 50 percent of pods in report [3], because saturation appeared only once the flag reached 100 percent of pods. | by design analysis |
| Missing index after migration | post-deploy | Canary checks watch request traffic, so they miss a dropped index that first slows background jobs while web requests still look normal. | by design analysis |
| Rate limit misconfigured | post-deploy | A canary that tracks only 5xx error rates misses a rise in 429 and 403 responses, which are the status codes the misconfigured limiters returned. | by design analysis |
| Schema drift between services | post-deploy | A canary error-rate gate stays silent when 404 responses between services have no alert, as in the portal report. | by design analysis |
| All sessions invalidated | post-deploy | A canary flags the logout surge only if token validation errors reach logs and alarms, so suppressed validation errors leave it blind, and the mixed old and new fleet it creates is itself the condition that splits sessions. | by design analysis |
| Background job queue backlog | post-deploy | Canary checks watch request traffic and miss stopped recurring jobs, and in report [1] no deploy failed while scheduled work had halted. | by design analysis |
| Container image base change | post-deploy | A canary needs a staged rollout, and jobs that pulled the upstream 24.0.7 or latest tag received the new base directly with no rollout to stage. | by design analysis |
| Disk full from new log volume | post-deploy | A canary judged only on error rate passes, because root partition usage after the operating system update climbed for about 8 hours before the first timeout. | by design analysis |
| DNS change propagation outage | post-deploy | A canary that compares only public request errors misses the fault, because the misconfigured cache load balancer broke internal traffic that public requests never carry. | by design analysis |
| Flag enabled for wrong segment | post-deploy | A canary compares error rates, and the wrongly targeted flag changed application behaviour without failing outright, so no metric marked the affected solutions. | by design analysis |
| Release marker missing in monitoring | post-deploy | Canary checks judge only the traffic the canary receives, and the post-deployment migration that removed the index ran outside the canary, so draining the canary did not improve the metrics. | by design analysis |
| Checkout blocked for segment | post-deploy | A canary that watches only 5xx errors misses the 4xx failures of the temporary-token workflow, which is one slice of gateway traffic. | by design analysis |
| Cold start latency regression | post-deploy | A canary that compares request duration without separating init duration blends the cold start increase from the SDK's startup import into warm invokes that do not pay it. | by design analysis |
| Config change without restart semantics | post-deploy | A canary that watches error rates misses a fault that shows as a traffic drop, which is how the port forwarding failure appeared. | by design analysis |
| Crash on older OS version | post-deploy | A phased store rollout shows the launch crash only when its first cohort includes devices on iOS 17 or older. | by design analysis |
| Double charge after deploy | post-deploy | The processing errors were isolated to customers on the Ruby client library, so a canary whose traffic share holds no such customers never sees the error rise. | by design analysis |
| Environment mismatch: production points to staging | post-deploy | The canary alert fires only after a share of production traffic already reaches the staging script, so that share sees HTTP 401 errors before rollback. | by design analysis |
| Error budget burned by retry storm | post-deploy | A canary window that closes before retry amplification starts passes the timeout change, and amplification began at 17:12 UTC after the 17:02 UTC deploy. | by design analysis |
| Flag default flips on missing config | post-deploy | A canary that receives no EU traffic does not see the gated chart formula feature, and a gated feature returns no error for the canary metrics to compare. | by design analysis |
| Push notification token reset | post-deploy | A canary at 2 percent of daily campaign volume produced bounces that went unnoticed until the next morning, because alerting did not catch this error at low volume. | by design analysis |
| Rolling deploy mixed versions | post-deploy | A canary that compares error rates on the web canary fleet misses failures that surface in background workers still running the old code, where the stuck pipelines were processed. | by design analysis |
| Third-party rate limit hit | post-deploy | A canary compares a new code version with the old one, and the region onboarding that throttled the authentication layer was not a code release, so the canary had no traffic to compare. | by design analysis |
| Thundering herd on restart | post-deploy | A canary that replaces a few pods sends only a small reconnect wave, so the database connection saturation in report [2] appears only at full deployment scale. | by design analysis |
For cited rows the number shows how many reports on that incident page name this class.
Only on this page
Canary and progressive delivery checks are cited most often for Memory leak in new release: 1 public report on that page name this class. The limitation stated there: The canary ring in report [1] did not expose the authorization service to enough traffic to discover the leak, so the build graduated to production at 6:37 PM UTC.
Catalogue entries that name this class
From their own documentation or site, with the verification date. Alphabetical. Not a recommendation.
Tools whose documentation names these test classes
- Argo Rollouts canary source verified 2026-10-21
- Flagger canary source verified 2026-10-21
- GrowthBook canary source verified 2026-10-21
- Kayenta canary source verified 2026-10-21
- LaunchDarkly canary source verified 2026-10-21
- Spinnaker canary source verified 2026-10-21
- Unleash canary source verified 2026-10-21
Providers whose site names these test classes
No provider in the catalogue names this test class on its site.
Questions
What is Canary and progressive delivery checks?
Canary and progressive delivery checks: a small share of traffic runs the new version with automated metric comparison. They run at the post-deploy stage.
What does Canary and progressive delivery checks catch?
Canary and progressive delivery checks catch regressions visible in traffic metrics, memory growth and error rate rise. 26 published incident types list this class.
What can Canary and progressive delivery checks not catch?
Canary and progressive delivery checks cannot catch faults in traffic the canary never receives, batch jobs and scheduled tasks.
How is Canary and progressive delivery checks different from other test classes?
Canary and progressive delivery checks share the post-deploy stage with end-to-end tests, configuration and environment tests, synthetic monitoring and chaos and resilience tests; they differ in what each catches.
Can Canary and progressive delivery checks run in a production environment?
Yes. Canary and progressive delivery checks run at the post-deploy stage, against production.
Which incidents is Canary and progressive delivery checks used for?
all sessions invalidated, app store rollback not possible, autoscaling policy misconfigured, background job queue backlog, checkout blocked for segment, cold start latency regression, config change without restart semantics, connection pool exhausted, container image base change, crash on older OS version, disk full from new log volume, dNS change propagation outage, double charge after deploy, environment mismatch: production points to staging, error budget burned by retry storm, flag default flips on missing config, flag enabled for wrong segment, memory leak in new release, missing index after migration, push notification token reset, rate limit misconfigured, release marker missing in monitoring, rolling deploy mixed versions, schema drift between services, third-party rate limit hit and thundering herd on restart. 1 of 26 links are cited in a public report; the rest are by design analysis.