Feature flags and progressive delivery in 2026: Statsig vs LaunchDarkly vs GrowthBook

The rollout was supposed to touch 10 % of users. The engineer set the percentage in the dashboard, hit save, and went to lunch. Two hours later, customer success pinged the engineering channel: "Getting a lot of EU complaints about the new checkout flow. Did something change?" A quick audit showed that 87 % of EU users had been placed in the treatment bucket. North American users were at 3 %. The overall number in the dashboard still read 10 % — because it was 10 %, averaged across the full user base. But user IDs in this system were assigned chronologically, and EU users had joined the platform first. The low-range numeric IDs clustered in Europe. The hash function bucketing users into the flag partition was applied to raw user IDs without salting. The result was a deterministic, invisible skew that looked correct in aggregate and was catastrophically wrong by geography.

Nobody had checked the assignment distribution by region before the rollout. Nobody had an automatic check that would have caught it. The "10 % rollout" number in the dashboard was technically accurate and practically useless.

This story is not unusual. Feature flags are, by 2026, table-stakes infrastructure for any team shipping continuously. The concept is simple enough that most teams implement their first version in a weekend. But the surface area of ways to get them wrong — cohort skew, phantom exposures, SRM, scope conflicts, improper decommissioning — is large enough that entire companies exist to handle just this problem. Statsig, LaunchDarkly, and GrowthBook are three of the most widely deployed options, each with meaningfully different architectures, statistical capabilities, and cost structures.

This post is a practitioner guide to the mechanics of feature flags and progressive delivery: how the systems work under the hood, where they fail, and how to choose between the major platforms in 2026. It assumes you have shipped at least one feature flag to production and have some of the resulting scars.

What's actually happening

The four types of flags — and why conflating them costs you

Every feature flagging system exposes roughly the same UI: create a flag, target some users, turn it on. The abstraction hides a set of fundamentally different use cases that have different operational requirements.

Kill switches are pure operational controls. A kill switch is a boolean that turns a feature on or off with no statistical analysis, no bucketing algorithm, and no experiment tracking. You use kill switches to disable a broken payment provider, revert a CSS regression, or dark-launch a feature that you can enable for any subset of users at any time. Kill switches must be fast to evaluate — latency in the evaluation path of a flag that controls whether your app is functional is a reliability problem — and they must be operationally reliable when your system is degraded. This creates a tension with platforms that rely on network calls to evaluate flags in real time.

Progressive rollouts are operational controls with a bucketing algorithm. Instead of on/off, you target a percentage of users and gradually increase that percentage over time. The percentage is applied consistently — the same user always gets the same bucket, because the bucket is determined by a hash of the user identifier. The hash must be salted per-flag to avoid the cohort skew problem described in the lede. If you use the same hash function on the same user ID for every flag, all flags will assign the same users to treatment, and your rollouts will not be independent.

Most platforms use a variation of MurmurHash or xxHash over "${flagName}:${userId}" to produce deterministic, flag-independent bucket assignments. This is the detail that the EU skew bug violated: the bucketing was applied to a raw user ID without the flag name as a salt, so the assignments were correlated across all flags and happened to be correlated with user geography.

Targeting rules are conditional routing. A flag returns different values based on attributes of the user or request: internal employees get the beta, users on the Starter plan get the old UI, users in the DE locale get a GDPR-specific consent flow. Targeting rules are how you handle legal, localisation, and plan-gating requirements that are not statistical experiments. They are also the mechanism by which you implement segment-based safety valves during progressive rollouts: "serve the new checkout to all users except users on the Enterprise plan and users in the EU until legal review completes."

Experiments are randomised controlled trials implemented via flags. A user is assigned to a variant (control or one of N treatments), the variant assignment is logged as an exposure event, the user interacts with the product, and a downstream metric (conversion, session length, revenue) is measured against that exposure event. The statistical engine then tests whether the observed difference between variants is larger than what you would expect by chance.

Experiments have requirements that kill switches and rollouts do not: you must log exposures correctly, maintain assignment consistency across sessions (a user who sees variant A on visit 1 must see variant A on visit 2), avoid leaking the assignment to UI elements that could bias behaviour before the metric event occurs, and run the experiment long enough to achieve the power you need given your traffic volume and expected effect size. Cutting an experiment short because the dashboard looks good is one of the most common statistical mistakes in industry experimentation.

The operational difference that matters: kill switches and rollouts have no statistical validity requirements. Experiments do. Mixing them in a single flag — running a rollout of a feature, then retrospectively calling the rollout an A/B test — produces a result you cannot trust, because the rollout process may have introduced novelty effects or selection bias that corrupt the causal estimate.

How SDK evaluation works: server-side, client-side, and edge

The three SDK modes represent meaningfully different performance and security trade-offs.

Server-side evaluation downloads the full ruleset for all flags from the feature flag service and evaluates flags locally on each request. The LaunchDarkly server-side SDK, for instance, downloads the complete flag configuration on initialisation and re-syncs it via a streaming connection. Each evaluation is a pure local function call — user + ruleset → variant. There are no network calls on the hot path. The trade-off is that the full ruleset, including your targeting rules and any user segmentation logic, lives on your server. You cannot safely expose the server-side SDK key to a browser, because anyone who captures it can query your entire flag configuration.

Client-side evaluation moves the evaluation to the flag service. The client SDK sends the user context to the LaunchDarkly evaluation service, which applies the rules and returns only the resolved values for that specific user — not the rules themselves. The client stores the resolved values locally and evaluates flags from that local cache without further network calls on the hot path. The security model is inverted: the client never sees your rules, only the results. The cost is a network round-trip at initialisation to hydrate the client cache. This round-trip is often in the 50–200ms range, which is tolerable if it happens once at app load but disastrous if it happens in the critical render path.

The Statsig client SDK initialisation follows a similar model. When you call StatsigClient.initializeAsync(), the SDK fetches the current flag assignments for the provided user context from Statsig's servers and caches them. Subsequent calls to checkGate() or getExperiment() are synchronous local lookups against that cache. Statsig also offers a bootstrap mode where the server pre-renders the assignment payload and passes it to the client at page load, eliminating the client-side network round-trip entirely. This is the right mode when you are rendering server-side and can afford the extra work in the SSR layer.

Edge evaluation is the newest mode and the one with the most implementation variance across platforms. In the LaunchDarkly model, edge SDKs push the full flag ruleset into CDN edge storage (Cloudflare Workers KV, Akamai EdgeKV, Fastly KV Store, or Vercel Edge Config), and the evaluation happens in the edge worker on each request without touching the flag service at all. LaunchDarkly's edge SDKs support Cloudflare, Akamai, Fastly, and Vercel as edge integration targets. The latency profile is excellent — evaluation happens milliseconds from the user, before the request reaches your origin — but the operational complexity is higher: you must ensure that flag changes propagate to edge storage quickly, and you must reason about what happens when the edge worker receives a request while a flag update is in transit.

GrowthBook's edge story is different. Because GrowthBook is primarily a warehouse-native analysis layer rather than a delivery network, its edge offering is thinner. The GrowthBook JavaScript SDK can operate with a cached payload fetched from GrowthBook's CDN at app initialisation, but it does not have the CDN-edge-worker integration depth that LaunchDarkly offers.

The phantom exposure bug

Exposure logging is the mechanism by which the system records that a specific user was assigned to a specific variant of an experiment. Without accurate exposure logs, the statistical analysis cannot tell you whether the observed difference in metrics is caused by the treatment or by the fact that your treatment and control groups were not actually comparable.

The phantom exposure bug is when you log an exposure for a flag evaluation that the user never actually experienced in the UI. It happens in several common patterns:

The most common pattern: you call checkGate() to determine which code path to execute, but you call it inside a function that runs every time any page loads, and you call it before checking whether the relevant UI is even visible in the current context. A user visits the settings page. Your codebase evaluates the new_checkout_flow gate on every page load for analytics prefetching. The user is assigned to treatment, the exposure is logged, but the user never visited the checkout at all. When the experiment results come in, the treatment group looks worse — because it is contaminated with users who were assigned to the variant but never experienced it, diluting the signal.

Statsig's server SDK provides explicit control over this: you can call checkGate(user, 'flag_name', { disableExposureLogging: true }) to evaluate the flag for routing purposes without logging an exposure, and then call Statsig.manuallyLogGateExposure(user, 'flag_name') exactly at the point where the variant is shown to the user. This two-phase pattern — evaluate first, log exposure at render — is the correct model for any experiment where the flag evaluation and the user experience are not colocated in the call stack.

GrowthBook's tracking callback model implements the same principle differently. The SDK's trackingCallback fires when isOn() or getFeatureValue() is called and resolves to an experiment variant, but you control when that resolution happens. If you call getFeatureValue() inside a React render that is conditionally executed, the tracking callback fires only when the component actually renders — which is the correct semantics. If you call it eagerly at app bootstrap for all possible flags, you recreate the phantom exposure problem. The discipline is yours, not the SDK's.

Cohort skew and targeting pitfalls

The EU skew scenario from the lede is an instance of a broader class of targeting pitfall: the assignment mechanism produces groups that appear balanced in aggregate but are imbalanced along a dimension that is correlated with both the treatment and the outcome.

Geographic clustering of user IDs is one cause. Another is time-of-day. A rollout that launches at 10 AM UTC and targets "the first 10 % of users to arrive" is not a random sample — it is a sample of users in timezones where 10 AM UTC corresponds to active working hours, which skews toward Europe. A 10 % percentage rollout based on user ID hash avoids this, but only if the hash function is applied correctly.

A second common cohort skew pattern is new user bias. New users who join the platform after a flag is turned on are 100 % in the treatment group (if the flag is fully on for new signups). Existing users may have a different rollout percentage. If you measure an outcome metric that is correlated with account age — retention at 30 days, purchase conversion — and your experiment coincides with a growth spike, your treatment group will contain a higher proportion of new users, and your result will partially measure the new-user effect rather than the treatment effect. The fix is to filter the experiment analysis to users who were assigned to a bucket before a specific cutoff date, which most experiment platforms support as a "user cohort" filter.

A third pattern is feature flag scope conflicts. If two flags modify the same UI component — say, one flag changes the CTA button colour and another changes the CTA button text — users in the intersection of both treatment groups receive an undefined combined state that neither flag was designed to produce. The correct tooling response is mutual exclusion: flag management platforms allow you to define exclusion groups such that a user is enrolled in at most one of a set of overlapping experiments. LaunchDarkly and Statsig both support mutual exclusion natively. GrowthBook calls this feature namespace partitioning.

The pattern

Flag type × platform × team size: how the decision space actually works

The decision about which feature flag platform to use is not primarily a features decision. It is a build vs buy, self-host vs managed, and cost-vs-control decision, and the right answer varies significantly by team size and experimentation maturity.

Small teams (fewer than 20 engineers) with no dedicated data team: the statistical capabilities of the platform matter less than simplicity and cost. Statsig's free tier — 2M events per month, unlimited flag checks, unlimited seats, A/B testing, and a 1-year analytics retention window — is generous enough to cover most small SaaS products entirely. You do not need warehouse-native analysis if you have no data warehouse. Statsig's built-in Pulse analytics product gives you experiment results without a separate BI layer.

Growth-stage teams with a data team and a BigQuery or Snowflake deployment: the calculus shifts. GrowthBook's warehouse-native model — where all experiment analysis queries run directly against your existing data warehouse — becomes attractive because it eliminates a separate telemetry pipeline. You already trust your warehouse data. Running experiment analysis there means you can join experiment assignments against revenue data, support ticket data, and any other metric stored in the warehouse, without exporting data to a third party. GrowthBook's MIT-licensed core means you can self-host the entire stack and pay nothing for the feature flag delivery layer, paying only for compute in your warehouse.

Enterprise teams with compliance requirements: LaunchDarkly's position is strongest here. Enterprise-grade RBAC, approval workflows, scheduled flag changes, and a fifteen-year track record in large regulated environments matter when the alternative is a compliance audit finding about "unauthorised access to production configuration." LaunchDarkly's Guardian tier adds automated rollback and guardrail metrics — the platform can detect that an error rate is increasing and roll back the flag without human intervention.

The progressive rollout ramp: mechanics and monitoring

The operational standard for progressive rollout in 2026 is a gated ramp with automated health signals at each gate. The manual version of this looks like:

• 1 % for 15 minutes — watch error rate and p99 latency
• 5 % for 30 minutes — watch the same signals
• 10 % for 1 hour — add business metrics (conversion, activation)
• 50 % for 4 hours
• 100 %

The manual version requires an engineer watching dashboards. The automated version wires the flag management platform to your observability system so that the rollout pauses or reverses automatically if a metric crosses a threshold.

LaunchDarkly's Experimentation product and the Guardian tier make automated rollback explicit: you define a guardrail metric (error rate below 1 %, p99 below 500ms), and if the metric is breached during the rollout, the platform can halt the ramp and optionally roll back. Statsig's Autotune feature offers a related capability via multi-armed bandit: instead of fixed ramp percentages, the platform allocates more traffic to the variant that is performing better on a defined metric, effectively auto-optimising the rollout.

The integration with observability is where most teams fall short. Honeycomb and Datadog both support feature flag tracking — you can emit the current variant assignment as a span attribute or tag on every trace. When you do this, filtering your latency histogram to feature_flag.new_checkout_flow = treatment gives you the p99 for exactly the users experiencing the new code path. Without this integration, you are comparing aggregate latency during the rollout period against the pre-rollout baseline, which conflates the effect of the flag with time-of-day effects, deployment effects, and traffic composition changes.

The correct integration pattern is:

import { trace } from "@opentelemetry/api"
 
const span = trace.getActiveSpan()
const variant = statsig.checkGate(user, "new_checkout_flow")
if (span) {
  span.setAttribute("feature_flag.new_checkout_flow", variant ? "treatment" : "control")
}

This adds the variant assignment as a span attribute on every request that evaluates this flag. Your observability platform can then segment any downstream metric by variant — not just latency, but database query count, downstream API calls, and any other span attribute you collect.

Eppo and the newer experimentation-layer entrants

Statsig, LaunchDarkly, and GrowthBook are the three platforms this post focuses on, but the competitive landscape has expanded. Eppo positions itself as a warehouse-native experimentation layer that does not include feature flag delivery — it provides the assignment and analysis layer and expects you to use your existing flag delivery mechanism. This is a meaningful architectural distinction: Eppo is explicitly not trying to replace your feature flag infrastructure, it is trying to be a better statistics engine on top of it. For teams that already have an opinionated view of flag delivery and want to upgrade their experiment analysis, Eppo is worth evaluating. For teams starting from scratch, the additional vendor relationship adds complexity without obvious benefit.

Split.io (now merged with Harness) is the fourth major managed option, positioned similarly to LaunchDarkly with a strong emphasis on the correlation between flag changes and metric changes (they call this "Data Hub" or "Metrics Impact"). The Harness acquisition means Split's future roadmap is more complex to predict.

The fix / approach

1. Audit your flag inventory before adding a platform

   Most teams that evaluate a new feature flag platform already have some flags. Before migrating or layering a new platform on top, audit the current state.

   The audit has three questions for each flag:

   1. Is this flag still live? Flags that wrapped a feature launch 18 months ago and were never cleaned up are dead code with a runtime cost (an SDK call that always returns the same value) and a cognitive cost (an engineer reading the code does not know if this is intentional). The industry term is "flag debt." A study by LaunchDarkly's developer relations team found that a typical engineering organisation has flags that are months or years past their intended lifespan. Every such flag is a branch that can interact with new flags in unexpected ways.

   2. Is this flag an experiment, a rollout, or a kill switch? If nobody knows, it is probably a kill switch that someone evaluated as an experiment, and the results are not trustworthy.

   3. Does the flag have an owner and a removal date? Flags without owners get orphaned. Flags without removal dates never get cleaned up. Encode both in the platform's metadata field or in a comment at the call site.

   After the audit, set a target: flags that have been fully rolled out to 100 % of users for more than 60 days should be removed from the codebase. The flag service can be kept alive as a record, but the code branch should be collapsed.

2. Instrument exposure logging at the right layer

   The canonical implementation separates the evaluation call from the exposure logging call, placing the exposure log exactly at the point where the user experiences the variant.

   In a server-side rendering context with Statsig:

   import Statsig from "statsig-node"
    
   // At request handling time — evaluate for routing, do NOT log exposure yet
   const isInTreatment = Statsig.checkGate(
     { userID: user.id, country: user.country, custom: { plan: user.plan } },
     "new_checkout_flow",
     { disableExposureLogging: true }
   )
    
   // Later, at the point of rendering the component that the user will see
   if (isInTreatment) {
     // Log exposure here — user is actually experiencing the variant
     Statsig.manuallyLogGateExposure(
       { userID: user.id, country: user.country, custom: { plan: user.plan } },
       "new_checkout_flow"
     )
     return renderNewCheckout(context)
   }
   return renderLegacyCheckout(context)

   In a GrowthBook client-side context, the equivalent is placing the isOn() call inside the conditional render:

   import { GrowthBook } from "@growthbook/growthbook"
    
   const gb = new GrowthBook({
     apiHost: "https://cdn.growthbook.io",
     clientKey: process.env.GROWTHBOOK_CLIENT_KEY,
     trackingCallback: (experiment, result) => {
       // This fires when isOn() resolves to an experiment variant
       // — only when the evaluation actually happens
       analytics.track("Experiment Viewed", {
         experimentId: experiment.key,
         variationId: result.key,
         variationName: result.name,
       })
     },
   })
    
   await gb.init({ streaming: false })
   gb.setAttributes({ id: user.id, country: user.country, plan: user.plan })
    
   // trackingCallback fires here, when this component renders
   const showNewFlow = gb.isOn("new_checkout_flow")

   The critical invariant: log exposures once per user per experiment assignment, at the moment the user experiences the variant, not when the flag is evaluated for routing. Any deviation from this invariant corrupts the experiment's denominator.

3. Validate assignment balance before and during rollouts

   Before declaring any experiment result valid, verify that the assignment ratios match the intended allocation. This is the SRM (Sample Ratio Mismatch) check.

   If you intend a 50/50 split and your platform reports 52 % control, 48 % treatment, you have an SRM. An SRM this large is almost certainly not random chance — it indicates a bug in the assignment mechanism, a filtering artifact in how you are counting assignments, or a survivorship bias (treatment users are more likely to complete the funnel and thus appear in your metric count but not in your assignment count).

   GrowthBook runs automatic SRM detection as part of its statistics engine, flagging experiments where the observed assignment ratio deviates from the intended ratio by more than would be expected by chance at p < 0.001. Statsig's Pulse dashboard surfaces assignment counts alongside results. LaunchDarkly's Experimentation product similarly surfaces assignment counts.

   The manual check is a chi-squared test on the assignment counts:

   // Validate that observed counts match expected proportions
   function checkSRM(
     controlCount: number,
     treatmentCount: number,
     expectedControlFraction: number
   ): { hasSRM: boolean; pValue: number } {
     const total = controlCount + treatmentCount
     const expectedControl = total * expectedControlFraction
     const expectedTreatment = total * (1 - expectedControlFraction)
    
     // Chi-squared statistic
     const chiSq =
       Math.pow(controlCount - expectedControl, 2) / expectedControl +
       Math.pow(treatmentCount - expectedTreatment, 2) / expectedTreatment
    
     // For 1 degree of freedom, p < 0.001 corresponds to chi-squared > 10.83
     const hasSRM = chiSq > 10.83
     // Approximate p-value using chi-squared CDF (simplified)
     const pValue = hasSRM ? 0.001 : 1 - Math.exp(-chiSq / 2)
    
     return { hasSRM, pValue }
   }

   Run this check before looking at any experiment results. An SRM means the results are not interpretable. Find the bug, fix it, reset the experiment.

4. Set up Statsig, LaunchDarkly, or GrowthBook for your stack

   The SDK integration patterns differ by platform and evaluation mode.

   Statsig server-side (Node.js):

   import Statsig from "statsig-node"
    
   // Initialise once at process start
   await Statsig.initialize(process.env.STATSIG_SERVER_KEY!, {
     environment: { tier: process.env.NODE_ENV === "production" ? "production" : "staging" },
     // Optional: custom logger
     logger: {
       warn: (msg) => logger.warn({ msg }, "statsig warning"),
       error: (msg) => logger.error({ msg }, "statsig error"),
     },
   })
    
   process.on("SIGTERM", async () => {
     await Statsig.shutdown()
   })
    
   // Per-request evaluation
   export function evaluateFlags(user: { id: string; email: string; plan: string }) {
     const statsigUser = {
       userID: user.id,
       email: user.email,
       custom: { plan: user.plan },
     }
    
     return {
       newCheckoutFlow: Statsig.checkGate(statsigUser, "new_checkout_flow"),
       checkoutExperiment: Statsig.getExperiment(statsigUser, "checkout_cta_copy"),
     }
   }

   GrowthBook server-side (Node.js):

   import { GrowthBook, setPolyfills } from "@growthbook/growthbook"
    
   // Fetch features once and cache — refresh on a schedule or via webhook
   let featuresPayload: Record<string, unknown> = {}
    
   async function refreshFeatures() {
     const res = await fetch(
       `https://cdn.growthbook.io/api/features/${process.env.GROWTHBOOK_CLIENT_KEY}`
     )
     const data = (await res.json()) as { features: Record<string, unknown> }
     featuresPayload = data.features
   }
    
   // Refresh on startup and every 60 seconds
   await refreshFeatures()
   setInterval(refreshFeatures, 60_000)
    
   export function createGrowthBookInstance(userId: string, attributes: Record<string, unknown>) {
     return new GrowthBook({
       features: featuresPayload,
       attributes: { id: userId, ...attributes },
       trackingCallback: (experiment, result) => {
         // Push exposure to your analytics pipeline
         void analytics.track(userId, "Experiment Viewed", {
           experimentId: experiment.key,
           variationId: result.key,
         })
       },
     })
   }

   LaunchDarkly server-side (Node.js):

   import * as ld from "@launchdarkly/node-server-sdk"
    
   const ldClient = ld.init(process.env.LAUNCHDARKLY_SDK_KEY!, {
     logger: ld.basicLogger({ level: "warn" }),
   })
    
   // Wait for client to be ready — this downloads the full ruleset
   await ldClient.waitForInitialization({ timeout: 10 })
    
   process.on("SIGTERM", () => ldClient.close())
    
   export async function evaluateFlag<T>(
     userId: string,
     attributes: Record<string, unknown>,
     flagKey: string,
     defaultValue: T
   ): Promise<T> {
     const context: ld.LDContext = {
       kind: "user",
       key: userId,
       ...attributes,
     }
     return ldClient.variation(flagKey, context, defaultValue) as T
   }

   The LaunchDarkly waitForInitialization call is important. If you start handling requests before the SDK has downloaded the ruleset, all flag evaluations return the default value — which could mean all users are served the control variant during your server's cold-start window. The timeout parameter (10 seconds in the example above) should be set to a value that is high enough to allow the network call but low enough that a dead LaunchDarkly service does not indefinitely block your server startup.

5. Wire feature flags into your progressive delivery pipeline

   The automated ramp pattern requires three components: the flag management platform, an observability signal, and a mechanism to update the flag percentage automatically.

   The simplest version uses a CI/CD pipeline step that runs after each deployment:

   // scripts/progressive-rollout.ts
   // Run as a GitHub Actions step with a cron or step condition
    
   import Statsig from "statsig-node"
    
   const RAMP_SCHEDULE = [
     { percentage: 1, holdMinutes: 15 },
     { percentage: 5, holdMinutes: 30 },
     { percentage: 10, holdMinutes: 60 },
     { percentage: 50, holdMinutes: 240 },
     { percentage: 100, holdMinutes: 0 },
   ]
    
   async function checkHealth(): Promise<boolean> {
     // Query your observability platform for error rate and p99
     const errorRate = await queryPrometheus(
       `sum(rate(http_requests_total{status_code=~"5.."}[5m])) / sum(rate(http_requests_total[5m]))`
     )
     const p99 = await queryPrometheus(
       `histogram_quantile(0.99, rate(http_request_duration_seconds_bucket[5m]))`
     )
     return errorRate < 0.01 && p99 < 0.5 // 1% error rate, 500ms p99
   }
    
   async function runRamp(flagName: string) {
     for (const { percentage, holdMinutes } of RAMP_SCHEDULE) {
       await updateFlagPercentage(flagName, percentage) // Platform API call
       console.log(`Ramp to ${percentage}% — holding for ${holdMinutes} minutes`)
    
       await new Promise((resolve) => setTimeout(resolve, holdMinutes * 60 * 1000))
    
       const healthy = await checkHealth()
       if (!healthy) {
         console.error(`Health check failed at ${percentage}%. Rolling back.`)
         await updateFlagPercentage(flagName, 0)
         process.exit(1)
       }
     }
     console.log("Ramp complete. Flag at 100%.")
   }

   This script is simplified — a production version would use a proper scheduler, retry logic, and alerting. But the shape is correct: automated health check at each ramp step, automated rollback if the check fails.

   The Statsig Management API, the LaunchDarkly REST API, and the GrowthBook REST API all support programmatic flag percentage updates. The LaunchDarkly PATCH /flags// endpoint accepts JSON Patch operations for updating rollout weights. Statsig's Management API supports updating gate rules via the /v1/gates/ PATCH endpoint.

Anti-patterns

Using feature flags as configuration management

The flag dashboard becomes attractive as a general configuration store. Someone adds a flag to control the maximum upload size. Another flag controls the rate limit threshold. A third controls which email provider is active.

This works until you have forty flags serving as configuration values, your flag service goes down, and your application cannot determine whether to use SendGrid or Postmark to send password reset emails. The flag service is now in the critical path for application functionality rather than just release control.

The distinction: feature flags govern which variant of a feature a user sees, for a limited time, during a rollout or experiment. They are not durable configuration. Configuration that is not expected to change frequently, or that represents permanent application settings, belongs in a configuration management system (AWS Parameter Store, Vault, environment variables with a proper secret management layer) rather than a feature flag service.

The practical rule: if a flag has been at 100 % for more than 90 days and there is no plan to change it, it is no longer a flag — it is a constant. Remove the flag branch from the code and, if the value needs to be configurable, move it to your configuration management layer.

Logging exposures for every flag on every request

A common performance anti-pattern is evaluating all flags for a user at request start and logging exposures for all of them simultaneously, regardless of which flags are relevant to the current request.

// Antipattern: logs exposures for all flags on every request
async function middleware(req, res, next) {
  const flags = await sdk.getAllFlags(user) // Evaluates and logs 40 flags
  req.flags = flags
  next()
}

This inflates exposure counts for every flag on every request path, making experiment numerators meaningless. The checkout flow experiment now has exposure events from users who visited the settings page, the documentation page, and the pricing page. Your conversion rate in the experiment will look terrible because the denominator is inflated with users who never reached the checkout.

The correct model: evaluate a flag only when the code path that uses it is actually reached. Log the exposure only when the user experiences the variant. If the checkout flow flag is only relevant on the checkout page, evaluate it only in the checkout route handler.

// Correct: evaluate and expose only when relevant
async function checkoutHandler(req, res) {
  const isNewFlow = sdk.checkGate(user, "new_checkout_flow")
  // Exposure is logged here — user is on the checkout page
  if (isNewFlow) {
    return renderNewCheckout(req, res)
  }
  return renderLegacyCheckout(req, res)
}

Deleting flags before removing the code

The operational temptation is to delete a flag in the dashboard as soon as you decide to fully ship a feature, then clean up the code later. If the code cleanup is delayed — as it often is — you now have checkGate("new_checkout_flow") calls in your codebase that return the default value (usually false) because the flag no longer exists in the platform. The feature effectively disappears for all users until someone notices.

The correct decommissioning sequence: first collapse the code branch (remove the flag call, ship the production code path as unconditional), then delete the flag in the dashboard.

LaunchDarkly flags support an archived state that allows you to mark a flag as fully shipped without deleting it from the SDK. Archived flags continue to return their last configured value if the SDK encounters them, rather than defaulting. This is a safety net for the race condition between code and dashboard cleanup.

Trusting aggregate rollout percentages without segment validation

The opening lede describes this in detail, but the anti-pattern deserves a concise statement: a 10 % rollout is only a 10 % rollout if the 10 % is a representative sample of your user base along every dimension that matters for your experiment.

Before running any experiment, validate the assignment distribution by at least:

• Geographic region (EU, US, APAC)
• Account creation date cohort (new users vs existing users)
• Plan tier (free, starter, pro, enterprise)
• Platform (web, iOS, Android)
• Traffic source (organic, paid, email)

If any of these dimensions shows a material imbalance between treatment and control, investigate before proceeding. The imbalance may be intentional (you are targeting a specific segment) or it may be a bug in the bucketing logic.

Most platforms support downloading a raw assignment export for this analysis. In GrowthBook, the assignment query runs directly in your warehouse, so you can join it against your user dimension tables to validate balance across any attribute. In Statsig, the Pulse dashboard surfaces breakdowns by custom user attributes. In LaunchDarkly, the Experimentation tab shows assignment counts; detailed breakdowns require exporting data to your analytics layer.

Running overlapping experiments on the same surface without mutual exclusion

Two product teams are running experiments simultaneously. Team A is testing a new CTA button colour on the checkout page. Team B is testing new CTA button copy on the same page. Users in the intersection of both treatment groups see the new colour and the new copy simultaneously. Users in control for both see the original colour and original copy. But users who are in treatment for one and control for the other see a mixed state.

If Team A's experiment shows a lift in conversion, is it because of the colour, or because of the interaction between the colour and the copy that Team B's control group saw? You cannot answer this from the data.

The fix is mutual exclusion (also called non-overlapping experiments or namespace partitioning). When two experiments touch the same user-facing surface, they must be defined in the same mutual exclusion group, so that each user is enrolled in at most one of them.

// GrowthBook namespace partitioning example
// Define experiments in non-overlapping namespace buckets
const ctaColorExperiment = {
  key: "cta-color",
  namespace: ["checkout-surface", 0, 0.5], // First 50% of namespace
  variations: [{ color: "blue" }, { color: "green" }],
}
 
const ctaCopyExperiment = {
  key: "cta-copy",
  namespace: ["checkout-surface", 0.5, 1.0], // Second 50% of namespace
  variations: [{ copy: "Buy Now" }, { copy: "Complete Purchase" }],
}

With this configuration, a user is placed into the checkout-surface namespace bucket, and that bucket determines which experiment (if any) they are enrolled in. A user in the first 50 % of the namespace range is only eligible for the colour experiment; a user in the second 50 % is only eligible for the copy experiment. No user sees both experiments simultaneously.

Case studies / numbers

Statsig post-acquisition: what the OpenAI deal means operationally

On September 2, 2025, Statsig announced it was joining OpenAI. The announcement, written by CEO Vijaye Raji, framed the acquisition as a continuation of Statsig's mission — "helping product teams build smarter and faster" — under OpenAI's umbrella. At the time of writing, the Statsig product continues to operate independently, with no announced changes to pricing, API compatibility, or SDK support timelines.

The practical implication for teams already on Statsig is limited in the short term. The SDKs continue to work, the pricing page is unchanged, and the platform continues to serve the existing customer base. The longer-term implication is more speculative: Statsig's expertise in exposure logging and experiment analysis is directly relevant to OpenAI's need to evaluate model changes and product features at scale. Statsig's tooling for measuring whether a new model response is better than the old one — which is structurally similar to a feature flag experiment, with the "variant" being the model version — is a likely area of product investment.

For teams evaluating Statsig in 2026, the acquisition introduces the standard acquisition-era uncertainty: roadmap shifts, pricing restructuring, and the possibility that the product's focus moves toward OpenAI's internal needs rather than the broader market. This is not a reason to avoid Statsig — many acquired developer tools continue to thrive for years after acquisition — but it is a factor to weigh against multi-year commitments.

A B2B SaaS team migrates from homegrown to GrowthBook

A 45-person B2B SaaS company had built a homegrown feature flag system four years earlier: a table in PostgreSQL, a simple admin UI, and a client library that fetched the flag table on application start. The system worked until they needed to run their first statistically rigorous A/B experiment. The homegrown system had no concept of user bucketing, no exposure logging, and no analysis layer.

They evaluated Statsig (cost: $150/month at the Pro tier for their usage volume), LaunchDarkly (cost: materially higher at their seat count and client-side MAU), and GrowthBook self-hosted (cost: compute in their existing GCP environment, roughly $40/month in Cloud Run and MongoDB Atlas).

They chose GrowthBook. The primary factors: data residency (they were bound by a customer agreement that restricted telemetry to their existing GCP environment), existing BigQuery deployment (GrowthBook's warehouse-native analysis ran directly against their existing event tables), and cost. The migration took two engineers three weeks, including migrating existing flag definitions, deploying the Docker container, and updating the client libraries.

The first statistically significant experiment they ran — a change to the onboarding email sequence — showed a 14 % lift in activation rate in the treatment group. The result was credible because GrowthBook's SRM check passed (assignment ratio matched the intended 50/50 split) and the experiment had run for the pre-specified two-week duration without early stopping. In their observation, this was the first experiment result the company had produced that the founders were willing to act on immediately rather than treating as directional evidence.

A consumer app uses LaunchDarkly for a global rollout with regional kill switches

A consumer mobile app with 2.3M monthly active users was rolling out a redesigned home feed algorithm. The new algorithm performed well in internal testing but the team was uncertain about behaviour at scale, particularly in markets where average session length was higher (where a worse algorithm would cause more harm per session).

They implemented the rollout using LaunchDarkly with a targeting rule that excluded the top three markets by session length (Japan, South Korea, Germany) from the initial rollout phases. The US and UK received the new algorithm at 5 %, then 20 %, then 100 % over 72 hours. Session length and D1 retention were monitored per-variant using Datadog feature flag tags.

When the US reached 100 %, they began the Japan rollout at 5 %. Within the first two hours, the monitoring showed a statistically significant reduction in D1 retention in the Japanese treatment group — 2.3 percentage points below control. The LaunchDarkly Guardian tier's guardrail metric had been set to a 1-point retention drop as the rollback threshold. The rollback triggered automatically, returning the Japanese market to 0 % treatment without manual intervention.

The engineering team received a PagerDuty alert: "LaunchDarkly Guardian rolled back home_feed_v2 for segment country=JP — D1 retention guardrail breached." This was the designed outcome. The experiment continued in the US and UK. The team spent two weeks diagnosing the Japan behaviour (a combination of longer average sessions amplifying a latency regression and a content coverage gap in the new algorithm's training data for Japanese-language content) before a corrected version was shipped.

In the team's retrospective, the key observation was: without automated rollback, the Japan incident would have run for an unknown number of hours before a human noticed the retention signal in the dashboard. In their experience, D1 retention signals typically took 24–48 hours to accumulate enough data to be noticed. The automated rollback limited the user impact window to under two hours.

The multi-armed bandit as an alternative to fixed allocation experiments

A content platform running a subject-line optimisation experiment for their weekly newsletter had a practical constraint: the newsletter went out once a week, and they had roughly 80,000 recipients. A traditional A/B test with 50/50 allocation and 80 % power to detect a 5 % open-rate lift would require several newsletter sends to accumulate sufficient data — meaning several weeks of sending the worse subject line to half the audience.

Statsig's Autotune feature implements a Thompson Sampling multi-armed bandit that dynamically shifts allocation toward the better-performing variant as data accumulates. In the first send, the allocation was 50/50. By the second send, the bandit had shifted to 65/35 in favour of the variant with higher early opens. By the third send, the allocation was 85/15.

The practical result: over four sends, the average open rate was higher than it would have been with fixed 50/50 allocation, because more recipients received the better subject line earlier. The trade-off is that the statistical analysis is more complex (bandit algorithms do not produce a clean frequentist hypothesis test result) and the result is harder to defend to stakeholders who expect a p-value.

Multi-armed bandits are the right tool when the cost of sending users to the worse variant is high relative to the learning value of maintaining balance. They are the wrong tool when you need a clean causal estimate that you can use to justify a decision to stakeholders who require statistical rigour.

Limits + caveats

Feature flags are not a substitute for automated testing

A flag that lets you roll back a bad deployment in 30 seconds is valuable. It is not a substitute for a test suite that would have caught the bug before it reached production. Teams that invest heavily in feature flags sometimes underinvest in pre-production testing because "we can always roll back." This reasoning is flawed: rollback works for deterministic bugs that are immediately visible. It does not work for bugs that corrupt persistent state (a bad database migration that creates inconsistent records), bugs that affect a small percentage of users (who may not generate enough signal to trigger the rollback), or bugs that are visible only in combination with another condition that requires hours of user interaction to trigger.

The correct relationship between flags and testing: flags reduce the blast radius of bugs that reach production. Tests reduce the rate at which bugs reach production. Both matter, and they are not substitutes.

Experiment analysis is only as good as the metric definitions

GrowthBook, Statsig, and LaunchDarkly all provide statistics engines that produce p-values, confidence intervals, and win probabilities. These numbers are only meaningful if the underlying metric — the thing you are measuring — is defined correctly and measured consistently.

Common metric definition problems: using sessions instead of users as the unit of analysis (sessions are not independent — a single user generates multiple sessions, violating the independence assumption of most statistical tests). Using a proxy metric that is not actually correlated with the outcome you care about. Including metric events from users who were exposed to the variant but then lost from the analysis due to attrition that is correlated with the treatment (if the treatment is harder to use, more users in the treatment group will abandon the flow, and their conversion events will be missing not at random).

Platform automation helps with some of these — GrowthBook's SRM detection catches the most common denominator bugs, and CUPED variance reduction addresses the correlated-noise problem — but it does not help with a fundamentally wrong metric definition. "We increased DAUs by 12 %" means nothing if DAUs are inflated by users who open the app and immediately close it after seeing a change they dislike.

Self-hosting GrowthBook has real operational costs

GrowthBook's MIT-licensed core is genuinely free to use. But "free to run" is not the same as "zero cost." Self-hosting requires a MongoDB instance, ongoing updates as GrowthBook releases new versions, security patches, backup management, and someone with enough knowledge to debug the system when it misbehaves. For a team with a strong platform engineering function, this cost is low. For a team where every engineer works on product, it is a meaningful ongoing burden.

The practical guidance: self-host GrowthBook if you have a data platform engineer or a DevOps function that can own the deployment. Use GrowthBook Cloud or a managed platform if you do not. The cost difference is real, but so is the operational overhead.

Flag evaluation at the edge introduces consistency challenges

Edge evaluation is fast — evaluation happens at the CDN edge, often within 10ms of the user. But edge nodes are eventually consistent. When you update a flag in the LaunchDarkly dashboard, the change propagates to edge storage via the CDN's replication mechanism, which is not instantaneous. During the propagation window (typically seconds to low minutes, depending on the CDN and cache TTL configuration), some edge nodes serve the old value and some serve the new value.

For kill switches, this is acceptable: a few seconds of inconsistency while a flag is being disabled is a reasonable trade-off for the performance benefit. For experiments, it can be a problem: if a user's first request hits an edge node with the old flag value and their second request hits an edge node with the new flag value, their variant assignment may change mid-session. This violates the assignment consistency requirement for experiments.

The correct policy: use edge evaluation for operational flags (kill switches, configuration), and use server-side evaluation (or edge evaluation with sticky sessions) for experiments where assignment consistency across a session is required.

Sources & further reading

The sources below were verified at time of writing. Documentation URLs for SDK-specific features change frequently; use the version number in the URL to pin to the documentation version you need.

Statsig feature flags documentation https://docs.statsig.com/feature-flags/overview The primary reference for Statsig's feature gate semantics, exposure logging control (disableExposureLogging), and Pulse analytics. Covers the free tier (2M events/month, unlimited flag checks, 1-year retention) and SDK initialisation modes.

Statsig OpenAI acquisition announcement https://statsig.com/blog/openai-acquires-statsig Vijaye Raji's announcement post, published September 2, 2025. Primary source for the acquisition context referenced in the case studies section.

Statsig Node.js server SDK https://docs.statsig.com/server/nodejsServerSDK Reference for server-side evaluation semantics, checkGate with disableExposureLogging, manuallyLogGateExposure, and flush behaviour. Code examples verified against the documented API.

LaunchDarkly SDK concepts: client-side vs server-side vs edge https://launchdarkly.com/docs/sdk/concepts/client-side-server-side The authoritative description of LaunchDarkly's three evaluation models. Explains why client-side SDKs receive only resolved values while server-side SDKs download complete rulesets. Covers edge SDK integrations for Cloudflare, Akamai, Fastly, and Vercel.

LaunchDarkly pricing tiers (2026) https://launchdarkly.com/pricing/ Source for the Developer (free), Foundation, Enterprise, and Guardian tier descriptions. Guardian tier automatic rollback on guardrail metric breach is documented here.

GrowthBook overview and platform documentation https://docs.growthbook.io/overview Primary reference for GrowthBook's modular architecture, warehouse-native experiment analysis, and SDK types. Covers the four feature types (boolean, number, string, JSON) and the override rule hierarchy.

GrowthBook statistics engine documentation https://docs.growthbook.io/statistics/overview Documents the Bayesian and frequentist statistical engines, sequential testing, CUPED variance reduction, SRM detection, and the uninformative prior default. The authoritative source for GrowthBook's stats model.

GrowthBook JavaScript SDK https://docs.growthbook.io/lib/js Reference for client-side and remote evaluation modes, the trackingCallback contract, streaming configuration, and the init() payload structure.

GrowthBook GitHub repository https://github.com/growthbook/growthbook Dual-licensing structure confirmed here: MIT for the core, GrowthBook Enterprise License for commercial features. Repository statistics (7.8k stars, 759 forks) verified at time of writing.

Statsig self-host (Warehouse Native) documentation https://docs.statsig.com/statsig-warehouse-native/introduction Documents the Warehouse Native offering (enterprise tier, analysis runs in customer's data warehouse), including CUPED, stratified sampling, and switchback test support.

OpenTelemetry feature flag semantic conventions https://opentelemetry.io/docs/specs/semconv/feature-flags/ The OTel specification for feature flag span attributes (feature_flag.key, feature_flag.variant, feature_flag.provider_name). The integration pattern for emitting flag assignments as span attributes follows this convention.

Statsig pricing page https://statsig.com/pricing Source for the free tier (2M events/month, unlimited flag checks, unlimited seats, A/B testing, 1-year analytics retention) and Pro tier ($150/month, 5M events included). Pricing verified at time of writing; usage-based model confirmed (charged on metered events, not MAU).