Release Incident Playbook

Release checklist for payment products, built from published payment incidents

Before a payment release, run checkout end to end for every customer segment, compute tax from the current rate table, and run each scheduled billing job once in staging. These checks come from 3 of the 4 published incidents in the payments and billing group, each graded likelihood low and impact high on 2 public reports. The 5 other incident types in that group stay withheld because fewer than 2 independent public reports were accepted for each.

6 cited public reports · verified 2026-10-21

Payment incidents with public reports

Checkout blocked for segment is a checkout step that fails for one customer segment, country or card type after a release.

Checkout blocked for segment carries the labels "Likelihood: low (2 public reports in 5 years)" and "Impact: high (2 of 2 reports: money)".

Tax calculation regression is tax or VAT computed with the wrong rate or jurisdiction after a rules or data update.

Tax calculation regression carries the labels "Likelihood: low (2 public reports in 5 years)" and "Impact: high (2 of 2 reports: money and compliance)".

Subscription renewal skipped is a scheduled renewal that does not run, or runs with stale plan data, after a billing job change.

Subscription renewal skipped carries the labels "Likelihood: low (2 public reports in 5 years)" and "Impact: high (2 of 2 reports: money)".

Double charge after deploy is a customer charged twice for one purchase after a release changes retry or idempotency behaviour.

Double charge after deploy carries the labels "Likelihood: not published (1 public report in 5 years)" and "Impact: high (2 of 2 reports: money)".

In-app purchase receipt validation fails, a mobile incident type flagged as payments, is a rejection of store receipts after a validation or sandbox endpoint change.

In-app purchase receipt validation fails carries the labels "Likelihood: low (2 public reports in 5 years)" and "Impact: high (2 of 2 reports: money)".

Payment incidents withheld for lack of public reports

The playbook publishes an incident page only on two independent public reports from different organisations, with at least one published in the last 5 years.

Payment webhook not processed is withheld with 1 of 2 required reports.

Wrong currency or amount is withheld with 0 of 2 required reports and none from the last 5 years.

Refund flow broken is withheld with 1 of 2 required reports and none from the last 5 years.

Promo code applied unlimited is withheld with 0 of 2 required reports.

Payout delayed by batch job is withheld with 0 of 2 required reports.

This guide gives no checklist items for the withheld types, because no accepted pair of reports backs them.

Checklist: checkout for every segment

End-to-end tests (user flows through the real UI against a deployed environment) list checkout failures per segment among what they catch.

End-to-end tests cannot catch rare data states, so a segment absent from the fixtures passes unnoticed.

Canary and progressive delivery checks (a small share of traffic runs the new version with automated metric comparison) compare the metrics they are given, and a 2024 checkout report shows 5xx alerts alone missing 4xx payment failures [2].

  1. Run the checkout flow end to end with one saved address per shipping country before each release.
  2. Add a unit test case for each country that falls back to a shared validation package when a validation package is bumped.
  3. Confirm the client-side checkout validation accepts every value the server-side check accepts, using one sample value per country.
  4. Place an end-to-end test payment through every token workflow the gateway integration supports before switching integration versions.
  5. Alert on the 4xx rate of payment requests alongside the 5xx rate during the canary.

Checklist: tax rates and tax data

Unit tests (tests of one function or class in isolation) list wrong calculations among what they catch.

Unit tests cannot catch data shape faults, so a stored invalid tax code needs an integration test.

Integration tests (real dependencies such as a database, queue or provider sandbox exercised together) cannot catch production data volume or production-only configuration.

  1. Compute expected tax amounts in unit tests from the current rate table entry, not from a literal multiplier.
  2. Search tax report formulas for literal rate values after each rate change and confirm each one matches the rate table.
  3. Save an item with a price override while the tax field is disabled in an integration test, then read the stored tax code back from the database.
  4. Confirm the endpoint that stores tax codes rejects an empty value instead of saving it.
  5. Generate one invoice per jurisdiction in staging and compare its tax rate with the rate table entry.

Checklist: scheduled renewals and billing jobs

Canary checks cannot catch faults in batch jobs or scheduled tasks, according to the playbook's test class definitions.

A scheduled renewal fault therefore needs a staging run of the job itself before release.

  1. Run the scheduled renewal job in staging against subscriptions whose renewal date is the run date.
  2. Compare the count of subscriptions due for renewal with the count the job processed in that staging run.
  3. Feed the renewal handler a gateway response with missing fields in a unit test and confirm it records a handled failure.
  4. List every scheduled billing job a system-wide change touches and run each one once in staging.
  5. Re-run the renewal job twice for the same date in staging and confirm no subscription is charged twice.

Worked example

A repository issue from 2026 reports that from release 11.2.0 a checkout block rejected four-digit Argentine postcodes after a validation package bump from 3.9.2 to 3.10.22 [1].

The 2026 issue states that the server-side check and the store API still accepted the same postcodes [1].

A merchant support ticket, not an alert, surfaced the 2026 checkout failure [1].

A status page report from 2024 states that a gateway integration move on 15 August 2024 broke transactions that used the temporary (CSE) token workflow [2].

The 2024 checkout report states that alerts covered 5xx errors and none covered the 4xx errors these transactions returned [2].

A status page report from 2025 states that a frontend update sent an empty array when a dropdown was disabled, which stored invalid tax codes [3].

The 2025 tax report states that a small percentage of invoices were generated with invalid tax information [3].

A maintainer fix from 2024 states that after the Estonian standard VAT rate moved from 20% to 22%, a tax report formula still multiplied by 0.2 [4].

A status page report from 2024 states that incomplete error handling in a recent change produced null pointer exceptions in subscription renewal requests through one gateway [5].

A status page report from 2026 states that an automated membership renewal job malfunctioned from 25 February 2026 after system-wide improvements [6].

The 2026 renewal report was posted on 28 February 2026 [6].

None of the 6 reports names a test class, so every caught-by entry on the three payment incident pages carries the by design analysis label.

Public reports cited

  1. [1]WooCommerce, 2026-10-09. The WooCommerce issue reports that from release 11.2.0 the Checkout block rejected four-digit Argentine postcodes after a validation package bump, so shoppers in Argentina could not place orders and merchants rolled back to 11.1.x. primary Checkout block rejects legacy 4-digit Argentine postcodes (e.g. 1213) since 11.2.0
  2. [2]Chargebee, 2024-08-16. Chargebee reported that after it moved its Adyen integration from the Classic to the Checkout integration on 15 August 2024, customer transactions through Adyen that used the temporary (CSE) token workflow failed with 4xx errors until the change was reverted. primary Payment Processing Issues With Adyen Gateway
  3. [3]Mews, 2025-09-17. Mews reported that a recently introduced Rate Management feature caused incorrect VAT or tax rates to be applied to reservation items with price overrides, and its postmortem traced this to a frontend update that stored invalid tax codes. primary Issue with Mews Reservation with wrong VAT
  4. [4]Odoo, 2024-03-25. An Odoo maintainer fix states that after the Estonian standard VAT rate changed from 20% to 22% at the start of 2024 and the Estonian tax report lines were updated, the report formula still computed tax at 0.2 of the taxable amount. primary [FIX] l10n_ee: tax report computes according to updated taxes
  5. [5]Chargebee, 2024-03-01. Chargebee reported that a recent change in its application caused collect-invoice and subscription renewal requests through the GoCardless gateway to fail with null pointer exceptions, and fixed the error handling. primary Service Interruption in the GoCardless Gateway
  6. [6]MATCHi AB, 2026-02-28. MATCHi reported that after recent system-wide improvements its automated membership renewal job malfunctioned from 25 February 2026, so some auto-renewals were not processed, and it deployed fixes and re-ran the job to restore affected memberships. primary Issues with automatic renewal of memberships

Questions

Which payment release incidents have public reports?

Five incident types flagged as payments have published pages: checkout blocked for segment, tax calculation regression, subscription renewal skipped, double charge after deploy and in-app purchase receipt validation fails. Each rests on 2 accepted public reports and carries impact high.

Why does the double charge after deploy page print Likelihood: not published?

Double charge after deploy is published on 2 accepted public reports, but only 1 of them falls inside the 5-year window. The page therefore prints "Likelihood: not published (1 public report in 5 years)" next to "Impact: high (2 of 2 reports: money)".

Does a canary catch a skipped subscription renewal?

Canary checks cannot catch faults in batch jobs or scheduled tasks. A 2026 status page report describes a membership renewal job that malfunctioned after system-wide improvements [6].

Which test catches a checkout failure for one country?

End-to-end tests list checkout failures per segment among what they catch, but only for segments present in their fixtures. A 2026 repository issue reports four-digit Argentine postcodes rejected from release 11.2.0 [1].