Skip to main content
Free Calculator

Unified A/B Test Calculator

Plan how many visitors you need and how long to run your test. Then check data quality and decide whether the result is statistically and practically meaningful.

Live plan

62,468 total observations

31,234 control · 31,234 variant

Traffic and statistical settings

Defaults are suitable for a standard fixed-horizon plan. Open these when your traffic, allocation, or pre-registered hypothesis differs.

Fixed-horizon plan

62,468 total observations

31,234 control · 31,234 variant

Control

31,234

Variant

31,234

Duration

7 days

Duration assumes 10,000 eligible observations/day at 100% exposure. Edit Traffic used for duration. Target 5.50% · alpha 5% · power 80% · two-sided. Switch to Analyze to carry this plan forward.

Preserve this plan before launch.

Carry the sample horizon, method, alpha, power, allocation, direction, stopping rule, and practical threshold into a private pre-test record.

Save this plan in GrowthLayer · free account

Share links store inputs in the URL fragment, which is not sent to the server. Anyone with the link can read those inputs.

Methodology

This calculator is explicit about its statistical contract. Conversion planning uses a large-sample two-proportion z approximation with the selected allocation. Conversion analysis uses a pooled z-test and a Newcombe-style effect interval built from Wilson score bounds. Average-metric planning uses the normal approximation, while analysis uses Welch's unequal-variance t-test and interval. SRM uses a chi-square goodness-of-fit check against the planned allocation. This is a fixed-horizon workflow. The SRM warning uses the declared strict threshold p < 0.001. Repeatedly checking and stopping when p crosses a threshold inflates false positives. It does not imitate a proprietary sequential or Bayesian engine, and it never relabels 1 − p-value as confidence. Large-sample methods can be unreliable for sparse conversions or highly dependent observations. Use an exact or cluster-aware method when those assumptions do not hold.

Frequently Asked Questions

Is this a sample size calculator or a significance calculator?
Both. Plan mode calculates sample and duration before launch. Analyze mode checks sample ratio mismatch, estimates the effect and its interval, reports the p-value, and compares the result with your planned sample and practical threshold.
What does a p-value mean?
It is the probability of seeing data at least this extreme if the null hypothesis and model assumptions were true. It is not the probability that the result happened by chance, and one minus the p-value is not confidence that the variant is better.
Should I use one-sided or two-sided?
Use two-sided by default because a variant can cause meaningful harm. Use one-sided only when the direction was declared before launch and an effect in the opposite direction would never support your decision.
Why does the SRM check appear first?
A sample ratio mismatch can point to assignment, instrumentation, or data-loss problems. If allocation is broken, a small p-value for the outcome does not rescue the experiment. Validate the data before interpreting the winner.
Why can another calculator return a different number?
Tools differ on test family, tail choice, continuity corrections, allocation, multiple-comparison handling, and whether they use fixed-horizon, sequential, or Bayesian inference. The important requirement is that planning and analysis use compatible assumptions.

Related Calculators

Focused Sample Size CalculatorFocused Significance CalculatorMinimum Detectable Effect CalculatorRevenue Impact Calculator

Choosing a method or double-checking another tool? Compare 11 public A/B test calculators and download the evidence matrix.

Updated for 2026. Built by GrowthLayer — built for evidence-aware experimentation teams.