What is A/B Test Sample Size?
This calculates how many visitors each variant needs before a result can be trusted, using the standard power calculation at 95% confidence and 80% power — the conventional defaults. Its real value is as a go/no-go filter run before the test, not after: if the required sample is larger than your traffic can supply in a reasonable window, the honest conclusion is that this test cannot be run, and you should test something bolder instead. Skipping this step is how teams end up confidently shipping changes based on noise.
Why it matters
- It prevents the single most common CRO mistake: calling a test 'done' before it has enough data to mean anything.
- It shows upfront whether your traffic level can realistically detect the effect size you care about.
- It stops the natural temptation to peek early and stop a test the moment one variant looks ahead.
The formula
n = (Z_α/2 + Z_β)² × [p₁(1−p₁) + p₂(1−p₂)] ÷ (p₂ − p₁)²
- Z_α/2, Z_β
- — fixed constants for 95% confidence and 80% power (1.96 and 0.84)
- p₁
- — baseline conversion rate
- p₂
- — baseline rate plus the minimum detectable effect
How to use this calculator
- 01Enter your current baseline conversion rate for the page or flow.
- 02Enter the minimum improvement, in percentage points, that would actually be worth acting on.
- 03The result is the sample size needed per variant — multiply by 2 for total test traffic needed.
- 04Use the A/B Test Duration calculator to see how long that will take at your current traffic level.
Worked example
A page converts at a 5% baseline, and the team wants to reliably detect at least a 1 percentage point improvement (to 6%).
- p₁ = 5%, p₂ = 6%
- Sample size ≈ 8,146 per variant
- Total traffic needed = 8,146 × 2 variants ≈ 16,300 visitors
- Halving the detectable effect to 0.5 points would roughly quadruple this requirement
This test needs roughly 8,146 visitors per variant (about 16,300 total) before the result can be trusted — smaller effects need dramatically larger samples.
Industry benchmarks
Compiled August 2026
Large effect (5+ point lift on low baseline)
Hundreds to low thousands per variant
Big changes are easier to detect with less traffic.
Moderate effect (1-2 point lift)
Several thousand per variant
Most realistic CRO test scenario.
Small effect (< 1 point lift)
Tens of thousands per variant
Often impractical without high traffic.
Multivariate / more than 2 variants
Multiply by each extra arm
Splitting traffic four ways needs roughly twice the total of a simple A/B.
Common mistakes
- Ending a test as soon as it 'looks' significant, before reaching the calculated sample size.
- Setting an unrealistically small minimum detectable effect for the traffic actually available.
- Running the calculation once and never re-checking it if baseline conversion rate shifts.
- Running many tests at once and celebrating whichever one hits significance — at 95% confidence, one in twenty tests produces a false winner by chance alone.
How to improve your sample size
Check realistic test duration before committing
A large sample size requirement may mean the test needs to run longer than is practical.
Calculate test duration →Test bigger changes if traffic is limited
Lower-traffic sites should test bolder changes with larger expected effects, which need much smaller samples to detect reliably.
Move the test earlier in the funnel
Testing against a higher-volume step — a landing page rather than a checkout confirmation — supplies the sample far faster, and the finding often applies downstream anyway.
Size the revenue at stake →Frequently asked questions
Why does a smaller effect need so much more traffic?+
Because the statistical noise around small differences is proportionally larger — detecting a subtle signal reliably requires far more data than detecting an obvious one.
What are 95% confidence and 80% power?+
Standard defaults: 95% confidence means a 5% chance of a false positive; 80% power means an 80% chance of detecting the effect if it's real. Both are conventional, widely used defaults.
Why does another calculator give a slightly different number?+
Because more than one standard formula exists. This uses the unpooled variance form; the pooled form returns a figure roughly 0.2% different on typical inputs. Neither is wrong, and a gap that small never changes whether a test is practical — but it is worth knowing before assuming one of the two tools is broken.
What if I genuinely cannot reach the sample size?+
Then do not run that test. The options are to test a bolder change, move to a higher-traffic page, or accept the decision is being made on judgement rather than data — which is a legitimate choice, as long as nobody later calls it evidence.
Why can't I just stop when it hits significance?+
Because checking repeatedly gives the data many chances to cross the line by luck. Peeking daily at a test with no real effect will still show significance surprisingly often — the sample size has to be fixed in advance for the confidence level to mean what it says.
Related calculators
Put this calculator on your site
Free to embed, no attribution required beyond the link the widget already carries. It posts its height to the parent window as marketing-calculators:height if you want to size the frame exactly.
<iframe src="https://markcelerate.com/embed/ab-test-sample-size-calculator" title="A/B Test Sample Size Calculator" width="100%" height="520" style="border:1px solid #ddd;border-radius:6px" loading="lazy"></iframe>