Rolling deploy mixed versions: pre-release and first 30 minutes checklists

Old and new versions serve traffic at once and disagree on a contract.

The pre-release checklist maps each item to a test class from the incident page. The first-30-minutes checklist covers detection, containment, the rollback decision and customer communication.

7 pre-release items · 7 first-30-minutes items · verified 2026-10-22

Pre-release checklist

PR-01Run the pre-production suite on a cluster where old and new application servers share traffic, and fail on any request that ends a logged-in session. integration
PR-02Sign a session cookie on the new version, send it to a server on the previous version and confirm the session stays valid, then repeat in the reverse direction. integration
PR-03Create a pipeline through a web node on the new version while background workers run the previous version, and confirm the pipeline leaves its first state. integration
PR-04Add a contract test that fails when the web code writes a pipeline state value that the deployed background worker version does not accept. contract
PR-05Add a contract test that fails when the session cookie signing algorithm changes while the previous release cannot verify cookies signed with the new algorithm. contract
PR-06Alert on session rejection errors split by application version within the first minute of a rolling deploy. canary
PR-07During the web canary window, track errors from the background worker that handles new merge requests and halt promotion when they rise above baseline. canary

First 30 minutes

FM-01Check whether failed requests began right after a rolling deploy started, and split the errors by application version.
FM-02Check whether the web fleet and the background workers run different versions, such as a web canary ahead of the workers.
FM-03Drain the canary fleet to stop new pipelines or sessions reaching the mixed-version path.
FM-04Decide on the rollback knowing it opens a second mixed-version window, and run it with the widest deployment parallelism the fleet allows.
FM-05Mark the faulty version unsafe so a pipeline unlock or a queued deployment cannot redeploy it.
FM-06If the automated rollback is blocked, run a manual rollback to the previous code.
FM-07Post a status update that names the failing surface, such as web sessions or pipeline status, and the surfaces that still work.

Download the checklist

Pre-release checklist (Markdown)1.6 KB · v1.0.0
First 30 minutes checklist (Markdown)1.1 KB · v1.0.0

Only in this checklist

PR-04 is the only item among 6 published checklists in this group that relies on contract tests: Add a contract test that fails when the web code writes a pipeline state value that the deployed background worker version does not accept.

Questions

What should be checked before a release to prevent Rolling deploy mixed versions?

7 pre-release items, PR-01 to PR-07. PR-01: Run the pre-production suite on a cluster where old and new application servers share traffic, and fail on any request that ends a logged-in session.

Is there a rollback plan for Rolling deploy mixed versions?

The first-30-minutes checklist has 7 items. FM-01: Check whether failed requests began right after a rolling deploy started, and split the errors by application version.

What should a team do in the first 30 minutes after Rolling deploy mixed versions is detected?

The first-30-minutes checklist has 7 items. FM-01: Check whether failed requests began right after a rolling deploy started, and split the errors by application version.

Is a Rolling deploy mixed versions checklist still needed if deployment is automated?

Yes. Automation runs the steps; the checklist names what the tests must cover: integration tests, contract tests and canary and progressive delivery checks.

Which pipeline stages does the Rolling deploy mixed versions checklist cover?

The pre-release items map to the pre-deploy, pre-merge and post-deploy stages.

In what formats can the Rolling deploy mixed versions checklist be downloaded?

Both checklists download as Markdown files: /downloads/rolling-deploy-mixed-versions-pre-release.md and /downloads/rolling-deploy-mixed-versions-first-30-minutes.md, licensed CC BY 4.0.