Synthetic monitoring: what they catch and the release incidents they cover

Synthetic monitoring (scripted checks of critical flows against production on a schedule) run at the post-deploy stage.

20 published incident types list synthetic monitoring as a class that catches them; 2 of these links are cited in a public report and 18 are by design analysis.

stage: post-deploy · 20 published incident types · 2 cited

What synthetic monitoring catch and cannot catch

Catches

  • checkout and login breaks
  • DNS and certificate failures
  • CDN misses

Cannot catch

  • faults behind authentication the script does not reach
  • rare segments

Both lists come from the site's test class definitions.

Incident types this class catches

IncidentStageLimitation on that pageBasis
Certificate expiredpost-deployA probe of public domains does not reach internal service-to-service or LDAP backend certificates, and the one probe that did fire 30 days ahead sent its alert to a channel that nobody acted on.cited in [2]
IAM policy change blocks servicepost-deployA synthetic check detects the failed export or package download only after the account-level block or bucket policy is live, so it shortens detection but does not stop the change.cited in [1]
SSO login brokenpost-deployA synthetic SSO login catches the break only for the identity provider configuration its script signs in with, so a fault limited to one certificate type or one SAML setup goes unseen.by design analysis
Provider API change breaks contractpost-deployA scheduled probe that does not call through a custom domain with iss validation, or does not parse the webhook with strict types, stays green because only those customer segments failed.by design analysis
Asset cache serves old bundlepost-deployA scripted login from one location can receive a fresh chunk while the stale chunk is served only to some browsers through other CDN edges.by design analysis
CORS policy blocks frontendpost-deployAn API-level synthetic check passes because direct API requests kept working; only a browser-based check sends the cross-origin request that fails.by design analysis
DNS change propagation outagepost-deployExternal DNSSEC probes went green after the DS record was published, because the failure hit only resolvers that had cached a negative answer for the wildcard name.by design analysis
OAuth callback URL not whitelistedpost-deployA scheduled login probe on the default domain stays green, because only applications on newly configured custom domains returned the invalid redirect URI error.by design analysis
Release marker missing in monitoringpost-deployA scripted upload through the HTTP router shows the 502 errors only if its file exceeds the edge worker memory buffer, because smaller uploads are a segment that passed.by design analysis
SDK upgrade changes behaviourpost-deployA storage provider's probe that keeps an older SDK version pinned keeps passing, because only clients on the new checksum defaults received the CRC32 header error.by design analysis
Analytics event schema brokenpost-deployA test event sent only through the legacy classic destination arrives intact, because the prefix came from the newer actions destination alone.by design analysis
API key rotation breaks integrationspost-deployA synthetic probe fails only after the bulk job has already disabled keys, so it shortens the outage but does not prevent a restore from a database backup.by design analysis
API version dropped while old app livepost-deployA scheduled probe that calls only the current API version stays green, because only old client versions received the HTTP 410 response.by design analysis
Critical path broken in one browserpost-deployA synthetic check scripted in one browser never loads the Files page in the affected browser, and customer support tickets were the first signal of the failure.by design analysis
CSP blocks required scriptpost-deployAn API-level synthetic check passes, because the health endpoint and login API kept returning 200 while the browser refused to run the page script.by design analysis
Deep link broken after routing changepost-deployA synthetic browser check follows the redirect to the new domain and passes, so it misses the iOS segment that rejects redirects for universal links.by design analysis
Encoding or timezone shiftpost-deployA synthetic check that runs as an account in the default timezone never sees the wrong results, because only accounts with a custom timezone are affected.by design analysis
Form submission silently failspost-deployA production probe that submits an unchanged form stays green, because the platform fault dropped only data in fields that had been added, removed or renamed.by design analysis
In-app purchase receipt validation failspost-deployA scheduled test purchase that buys only a subscription, or runs only on iOS, stays green while Android consumable purchases fail.by design analysis
Localisation strings missingpost-deployA synthetic script pinned to one locale misses a gap that affects only the Russian room list or only the pt-PT build, since each locale is a segment the script never visits.by design analysis

For cited rows the number shows how many reports on that incident page name this class.

Only on this page

Synthetic monitoring are cited most often for Certificate expired: 2 public reports on that page name this class. The limitation stated there: A probe of public domains does not reach internal service-to-service or LDAP backend certificates, and the one probe that did fire 30 days ahead sent its alert to a channel that nobody acted on.

Catalogue entries that name this class

From their own documentation or site, with the verification date. Alphabetical. Not a recommendation.

Tools whose documentation names these test classes

Providers whose site names these test classes

No provider in the catalogue names this test class on its site.

Questions

What is Synthetic monitoring?

Synthetic monitoring: scripted checks of critical flows against production on a schedule. They run at the post-deploy stage.

What does Synthetic monitoring catch?

Synthetic monitoring catch checkout and login breaks, DNS and certificate failures and CDN misses. 20 published incident types list this class.

What can Synthetic monitoring not catch?

Synthetic monitoring cannot catch faults behind authentication the script does not reach and rare segments.

How is Synthetic monitoring different from other test classes?

Synthetic monitoring share the post-deploy stage with end-to-end tests, configuration and environment tests, canary and progressive delivery checks and chaos and resilience tests; they differ in what each catches.

Can Synthetic monitoring run in a production environment?

Yes. Synthetic monitoring run at the post-deploy stage, against production.

Which incidents is Synthetic monitoring used for?

analytics event schema broken, aPI key rotation breaks integrations, aPI version dropped while old app live, asset cache serves old bundle, certificate expired, cORS policy blocks frontend, critical path broken in one browser, cSP blocks required script, deep link broken after routing change, dNS change propagation outage, encoding or timezone shift, form submission silently fails, iAM policy change blocks service, in-app purchase receipt validation fails, localisation strings missing, oAuth callback URL not whitelisted, provider API change breaks contract, release marker missing in monitoring, sDK upgrade changes behaviour and sSO login broken. 2 of 20 links are cited in a public report; the rest are by design analysis.