Pre- & Post-Test Calculators: Validate Tests Before and After
Plan With Confidence, Interpret With Precision
Stop wasting weeks on tests that never had a chance. GrowthLayer's calculators validate your sample size before you start and interpret results with statistical rigor after you finish.
The Real Cost of Bad Statistics
Underpowered Tests
80% of A/B tests are underpowered from day one. You'll never detect a real winner because you didn't have enough traffic.
Sample Ratio Mismatch
SRM bugs invalidate results silently. Without detection, you could be implementing "winners" based on broken data.
Peeking at Results
Checking significance daily inflates false positive rates. What looks like a 95% confidence "winner" might be noise.
Unrealistic MDE Goals
Hoping for 20% lift when your test can only detect 50%+ changes? You've set yourself up for "inconclusive" from the start.
Two Calculators. One Complete Testing Workflow.
Validate before you launch. Interpret after you finish. GrowthLayer covers both.
Statistical Methods You Can Trust
Z-Test
Standard two-proportion z-test with 1-tailed or 2-tailed options
P-Values
Accurate calculation using normal CDF approximation
Confidence Intervals
Wilson score intervals at 95% confidence
Bayesian Probability
Monte Carlo simulation for probability to beat control
SRM Detection
Chi-square goodness-of-fit to catch traffic allocation bugs
Effect Size
Cohen's h to measure practical significance
More than just a p-value
Get statistical significance, confidence intervals, sample ratio mismatch checks, and an AI-generated plain-English interpretation of what your results mean and what to do next.
Complete Your Testing Workflow
Calculate, analyze, and learn. Here's what comes next.
Frequently Asked Questions
How do I calculate A/B test sample size?
Our pre-test calculator uses power analysis to determine the required sample size based on your baseline conversion rate, minimum detectable effect (MDE), statistical power (typically 80%), and significance level (typically 95%). Enter your metrics and get an exact sample size plus estimated test duration based on your traffic.
What is statistical significance in A/B testing?
Statistical significance indicates the probability that your observed difference between variants isn't due to random chance. A 95% significance level means there's only a 5% chance the result is a false positive. Our calculator provides both frequentist (p-value) and Bayesian (probability to beat control) interpretations.
What is SRM and why does it matter?
Sample Ratio Mismatch (SRM) occurs when traffic isn't split as expected between variants—often due to bugs in experiment implementation. SRM can completely invalidate your results. Our calculator automatically detects SRM using chi-square tests and warns you before you trust bad data.
Should I use Bayesian or frequentist analysis?
Both approaches have merit. Frequentist gives you p-values and confidence intervals—good for binary "significant or not" decisions. Bayesian gives you probability statements like "87% chance B beats A"—more intuitive for stakeholder communication. GrowthLayer provides both so you can use what works best for your team.