An A/B test lets you run two or more cancel flows side by side, splitting your cancelling customers between them so you can see which flow saves the most subscriptions. Instead of guessing whether a discount beats a pause offer, you let real cancellation traffic decide. This article explains who can run A/B tests, how to set one up, how traffic is split, and how to read the results and pick a winner.
Before you start
A/B testing is a Pro feature, and there are two requirements before you can create a test:
Your billing provider must be connected. Open Integrations and connect Stripe first — without it the A/B Test page shows a connection prompt instead of your tests.
You need at least two cancel flows to compare. You can build these on the Configure page under Flows.
When both are in place, open Flows and choose A/B Test from the menu.
Creating a test
From the A/B Test page, start a new test to open the setup wizard. It has four steps.
Step 1: Details
Give your test a clear Test name (up to 100 characters) so you can recognise it later, for example "Discount offer vs. pause offer". You can also write an optional Hypothesis describing what you expect to happen.
Then choose when the test should end:
After a set duration — the test runs for a fixed number of days regardless of traffic (between 3 and 365 days).
After enough sessions — the test runs until it has collected a target number of sessions (between 50 and 100,000).
Step 2: Select flows
Pick 2 to 5 flows to compare. A few rules apply here:
All flows in one test must be the same type — either all Widget flows or all Page flows. Once you select your first flow, flows of the other type are disabled.
A flow that is already used in another running test can't be added — it's marked as in use.
You can include Draft flows. Any draft flow you select will be set to Live automatically when the test starts.
Step 3: Traffic split
Decide what share of cancelling sessions each flow receives. Choose Equal to divide traffic evenly, or Custom to set each percentage yourself. The split must add up to exactly 100% before you can continue, and a live bar shows the distribution as you adjust it.
Step 4: Review
Check the summary of your name, end condition, traffic split, and variants. You can optionally turn on Auto-declare winner: pick the metric (save rate, revenue saved, offer accepted, completion rate, or fewest offers declined) and when the test ends, the best-scoring flow is automatically named the winner — its flow stays live and the others are paused.
When you're ready, choose Save as draft to keep configuring later, or Start test to launch it immediately.
💡 Your other flows keep working
Only the flows you add to the test are affected. Any flows you didn't select continue running normally.
How traffic is assigned
Each cancelling customer is bucketed into one variant and stays there for the rest of the test, so a returning customer always sees the same flow. The assignment is based on the customer's identity (such as their subscription, customer record, or email) and is one-way and private — it can't be traced back to the individual.
Pausing, resuming, and stopping
From the three-dot menu on any test card you can manage its lifecycle:
Pause test — temporarily stops collecting new data. Pausing does not extend a duration-based schedule, so a paused test can still reach its end date.
Resume test — picks a paused test back up.
Stop test — ends data collection permanently. This can't be undone.
If stopping or pausing would leave two flows that target the same audience both live at once, you'll be asked to decide which one stays live and which is paused — only one flow per audience can run at a time.
Reading the results
Open a test to see its results. The Detailed stats table compares each variant across Sessions, Save rate, Offer accepted, Offer declined, Dismissed, Pending, Revenue saved, Average session length, and Completion rate, highlighting the best value in each row.
The Statistical significance panel tells you how confident you can be in the result:
Significant (95%+ confidence) — a clear winner.
Trending (80–94%) — one flow is ahead, but keep the test running for a more reliable result.
Not enough data — the front-runner is shown with a reminder to keep collecting sessions.
⚠️ Give the test time
For a trustworthy result, aim for at least 100 completed sessions per variant. Declaring a winner too early can lead you to the wrong conclusion.
Declaring a winner
Once a test is stopped (or running with enough data), you can Declare a winner. The results suggest the flow with the highest save rate, but you choose the final winner. When you declare it:
The winning flow is set Live.
Losing flows that target the same audience as the winner are always paused, since two flows can't serve one audience at the same time.
For losing flows that target a different audience, you decide whether to Keep live or Pause flow — they can keep running independently.
After a winner is declared the test is marked Completed, and you keep the full results for reference.