Industry estimates suggest most A/B tests are underpowered from day one. You will never detect a real winner if you do not have enough traffic to begin with. The Sample Size Calculator tells you up front whether your test is actually feasible.

Sample ratio mismatch

SRM bugs invalidate results silently. If your traffic split is 48/52 instead of 50/50, the populations may not be comparable — but most tools never warn you. The SRM calculator runs a chi-squared check on the actual visitor counts.

Peeking at results

Checking significance daily inflates false positive rates. What looks like a 95% confidence winner halfway through the test is often statistical noise. The Significance Calculator reports the fixed-horizon p-value without relabeling 1 − p as confidence that the variant wins.

Unrealistic MDE goals

Hoping for a 20% lift when your traffic level can only reliably detect 50%+ changes guarantees an inconclusive result. The MDE Calculator works backwards from your traffic to tell you the smallest effect you can actually distinguish from noise.

Statistical methods used

Every calculator on this page uses peer-reviewed statistical methods. No black-box approximations, no “simplified for marketers” shortcuts.

Z-test

Two-proportion z-test with one- or two-tailed options for conversion rate comparisons.

P-values

Computed via the normal CDF approximation, accurate at typical A/B test sample sizes.

Confidence intervals

Wilson score intervals at 95% confidence — better small-sample behaviour than Wald.

Bayesian probability

Monte Carlo simulation for probability the variant beats control given the data.

SRM detection

Chi-squared goodness-of-fit on actual visitor counts versus the intended split.

Effect size

Cohen's h for practical-significance reporting alongside statistical significance.

Want to track your test results, not just calculate them?

GrowthLayer is a free A/B test library that remembers every experiment your team runs.

Start free →