Flag enabled for wrong segment: pre-release and first 30 minutes checklists

A feature flag targets the wrong users, region or percentage after a rule edit.

The pre-release checklist maps each item to a test class from the incident page. The first-30-minutes checklist covers detection, containment, the rollback decision and customer communication.

8 pre-release items · 8 first-30-minutes items · verified 2026-10-21

Pre-release checklist

PR-01List the audience every changed flag rule resolves to and confirm internal-test flags resolve only to internal accounts, not to every first-party application. config
PR-02Evaluate each version-based flag rule against the targeted environment's version and confirm the solution's current app-builder version does not change the result. config
PR-03Simulate the weekly automatic upgrade of a development environment and confirm no flag changes state without a customer deploy. config
PR-04Run the new-user login flow on the Android and iOS apps with each changed flag set to its production audience. e2e
PR-05Create and edit a team with an external identity provider group selected, then reload the team page and confirm the group link was saved. e2e
PR-06Run the flagged feature's end-to-end tests with the flag on and off for an account in each targeted release channel. e2e
PR-07Roll a targeting change out to one release channel at a time and hold each step until that channel's support ticket volume stays at its baseline. canary
PR-08Compare a behaviour metric of the flagged feature, such as saved group links, between flagged and unflagged accounts before widening the rollout. canary

First 30 minutes

FM-01Check support tickets and user reports for behaviour changes or sign-in failures that started after a flag rule edit.
FM-02List the flags whose targeting rules or rollout percentages changed in the last day and the audience each one now resolves to.
FM-03Disable the suspect flag platform-wide or scale it back to 0 percent of actors.
FM-04Halt any further rollout of the flag to the remaining release channels.
FM-05Roll back code only if disabling the flag does not restore logins or saves.
FM-06Identify affected accounts from the flag evaluation records rather than from error metrics, because changed behaviour raises no errors.
FM-07Find data written while the flag was on, such as team edits saved without their group link, and ask affected customers to re-check those records.
FM-08Post a status update that names the affected app or feature, the platforms that were unaffected and the time the flag was disabled.

Download the checklist

Pre-release checklist (Markdown)1.8 KB · v1.0.0
First 30 minutes checklist (Markdown)1.3 KB · v1.0.0

Only in this checklist

PR-04 is the only item among 5 published checklists in this group that relies on end-to-end tests: Run the new-user login flow on the Android and iOS apps with each changed flag set to its production audience.

Questions

What should be checked before a release to prevent Flag enabled for wrong segment?

8 pre-release items, PR-01 to PR-08. PR-01: List the audience every changed flag rule resolves to and confirm internal-test flags resolve only to internal accounts, not to every first-party application.

Is there a rollback plan for Flag enabled for wrong segment?

The first-30-minutes checklist has 8 items. FM-01: Check support tickets and user reports for behaviour changes or sign-in failures that started after a flag rule edit.

What should a team do in the first 30 minutes after Flag enabled for wrong segment is detected?

The first-30-minutes checklist has 8 items. FM-01: Check support tickets and user reports for behaviour changes or sign-in failures that started after a flag rule edit.

Which configuration checks apply to Flag enabled for wrong segment?

PR-01: List the audience every changed flag rule resolves to and confirm internal-test flags resolve only to internal accounts, not to every first-party application. PR-02: Evaluate each version-based flag rule against the targeted environment's version and confirm the solution's current app-builder version does not change the result. PR-03: Simulate the weekly automatic upgrade of a development environment and confirm no flag changes state without a customer deploy.

Is a Flag enabled for wrong segment checklist still needed if deployment is automated?

Yes. Automation runs the steps; the checklist names what the tests must cover: configuration and environment tests, end-to-end tests and canary and progressive delivery checks.

Which pipeline stages does the Flag enabled for wrong segment checklist cover?

The pre-release items map to the pre-deploy and post-deploy stages.