Release Incident Playbook

Flag enabled for wrong segment: symptoms, cause, and the test that catches it

A feature flag targets the wrong users, region or percentage after a rule edit.

Flag enabled for wrong segment is documented by 3 accepted public reports from 2 organisations, 2 of them published in the last 5 years.

  • Likelihood: low (2 public reports in 5 years)
  • Impact: medium (1 of 2 reports: availability)

3 public reports · verified 2026-10-21

Symptoms

User-visible

  • The Android mobile app could not log in new users, while the iOS app was unaffected.[2]
  • Customers could not link teams to external identity provider groups on the create or edit team page.[3]
  • Team creation and edits appeared to succeed, but the selected group was not linked after the form was submitted.[3]
  • Some customers saw behavioural differences in their applications, while most customers saw no observable change.[1]

Monitoring-visible

  • Support tickets increased on the morning of 7 May.[1]
  • No monitoring signal marked the full set of affected solutions, because application behaviour changed without failing outright.[1]
  • Reports from users who could not sign in to the Android app came before the first public update.[2]

Root causes

  1. stage: config A feature under internal test was enforced through its flag for every first-party application, including the mobile app. A mismatch between client and server expectations then made mobile logins fail.[2]
  2. stage: config An experimental UI selection component with a bug was rolled out to customers through a feature flag by mistake.[3]
  3. stage: config A new version-based targeting rule evaluated each solution's current app-builder version instead of the version of the targeted environment. Development environments upgrade automatically each week, so flag activation followed the platform release schedule instead of customer deploys.[1]

Public examples

  1. [1]Appfarm, 2026-05-13. Appfarm added version-based targeting to its feature flag system for the Flows rollout, but the rule evaluated against each solution's Appfarm Create version instead of the targeted environment's version, so the flag activated in customer environments on the platform release schedule rather than on customer-initiated deploys until the flag was disabled platform-wide. primary Post-mortem: Flows feature flag incident, May 7, 2026
  2. [2]GitHub, 2025-06-20. A GitHub App feature under internal test, controlled by a feature flag, was enforced for all GitHub-owned applications including GitHub Mobile, and the Android app could not log in new users until the flag was disabled. primary Disruption with the GitHub mobile android application
  3. [3]GitHub, 2024-07-25. An experimental GitHub UI component was rolled out to customers through a feature flag by mistake, and team-to-external-group links silently failed to save until the flag was scaled back to 0 percent of actors. primary Linking internal teams to external IDP groups was broken for some users between 15:17-20:44 UTC

Test classes that catch it

Test classStageLimitationBasis
Configuration and environment testspre-deployA configuration test can show that an internal-test flag resolves to every first-party application, but it does not show that the resulting client and server mismatch breaks new-user logins.by design analysis
End-to-end testspre-deployAn end-to-end run before the rollout passes, because the version rule switched the flag on later, when the weekly automatic upgrade changed the version it evaluated.by design analysis
Canary and progressive delivery checkspost-deployA canary compares error rates, and the wrongly targeted flag changed application behaviour without failing outright, so no metric marked the affected solutions.by design analysis

Risk

  • Likelihood: low (2 public reports in 5 years)
  • Impact: medium (1 of 2 reports: availability)
Counts behind the grades
Accepted public reports2
Published in the last 5 years2
Impact stated: degraded1
Impact stated: availability1

How the grades are computed

Pre-release checklist

  • PR-01
  • PR-02
  • PR-03
  • PR-04
  • PR-05
  • PR-06
  • PR-07
  • PR-08

First 30 minutes

  • FM-01
  • FM-02
  • FM-03
  • FM-04
  • FM-05
  • FM-06
  • FM-07
  • FM-08

Download the checklist

What catches this: catalogue entries

Listed because their own documentation or site names configuration and environment tests, end-to-end tests, canary and progressive delivery checks on the verification date. Alphabetical. Not a recommendation.

Tools whose documentation names these test classes

Providers whose site names these test classes

Only on this page

Flag enabled for wrong segment in report [1] came from a targeting rule that read the wrong version number. The rule checked each solution's current app-builder version instead of the version of the environment it targeted [1]. Development environments upgrade automatically each week, so the flag switched on with the platform release schedule rather than with a customer deploy [1]. An increase in support tickets was the first sign [1].

Report [3] shows a second silent form of the same fault. Team edits appeared to succeed, but the selected identity provider group was not linked [3]. Neither report needed a code rollback: report [3] scaled the flag back to 0 percent of actors, and report [1] disabled it platform-wide.

Questions

What causes flag enabled for wrong segment?

In report [2], a feature under internal test was enforced for every first-party application, including the Android app. In report [3], an experimental UI component was rolled out to customers through a flag by mistake. In report [1], a version-based rule evaluated the wrong version number.

What are the symptoms of flag enabled for wrong segment?

The Android app could not log in new users while the iOS app was unaffected [2]. Team edits appeared to succeed, but the selected group was not linked [3].

Why did tests not catch flag enabled for wrong segment?

In report [1], the flag changed behaviour without failing outright, so no reliable signal marked the affected solutions. The flag switched on with weekly development environment upgrades, not with a deploy that a test suite runs against [1].

Which test class catches flag enabled for wrong segment, and what does it miss?

Configuration tests, end-to-end tests and canary checks are listed, all with the by-design label. A configuration test shows the audience a rule targets but not the login failure from a client and server mismatch.

Which public reports document flag enabled for wrong segment?

Three primary reports from two organisations document it. They were published between 2024-07-25 [3] and 2026-05-13 [1].

Should a team roll back or fix forward when flag enabled for wrong segment happens?

All three reports ended the impact by switching the flag off. Report [3] scaled the flag back to 0 percent of actors. Report [1] disabled the flag platform-wide at 10:47 and halted further rollout at 11:15.

Release Incident Playbook (2026). Flag enabled for wrong segment: symptoms, cause, and the test that catches it, content version 1.0.0. https://releasefailures.com/incidents/flag-enabled-for-wrong-segment/. Accessed 2026-10-21.