Check allocation, effect size, intervals, and the declared fixed-horizon result—then create an analyst-reviewed decision record.
Enter your observed results
Result fields start blank so example numbers cannot be mistaken for your experiment.
Statistical settings and original plan
Use the assumptions declared before launch. A plan created in this calculator is carried here automatically.
Ready for your results
Enter both arms above, or load the clearly labeled example, to generate a decision readout.
Share links store inputs in the URL fragment, which is not sent to the server. Anyone with the link can read those inputs.
Methodology
This is a fixed-horizon post-test workbench, not an automatic winner generator.
Conversion analysis uses a pooled two-proportion z-test as the declared decision rule and a Newcombe-style score interval as a companion boundary diagnostic. Average-metric analysis uses Welch's unequal-variance t-test and interval. Sample ratio mismatch is checked first against the allocation you enter.
A positive or negative fixed-horizon classification requires a supplied planned sample and that both arms reached it. Because a public calculator cannot verify when that plan was created, the result remains an analyst-reviewed readout rather than an automatically approved ship or stop decision. Without a plan, the readout is explicitly exploratory.
The calculator never treats 1 minus the p-value as confidence, never votes frequentist and Bayesian methods together, and never turns non-significance into proof of no effect.
Frequently Asked Questions
Why does GrowthLayer ask for the planned sample?▾
A small p-value does not prove that the stopping rule was valid. Supplying the sample target chosen before results lets the calculator check whether the fixed horizon was reached. Without it, GrowthLayer shows an exploratory analysis and requires human review instead of naming a winner.
Which statistical method makes the decision?▾
For conversion metrics, the declared decision rule is a fixed-horizon pooled two-proportion z-test. The score interval is a companion diagnostic. For average metrics, the declared method is Welch's unequal-variance t-test. GrowthLayer does not combine frequentist and Bayesian outputs into a synthetic confidence score.
Can I stop when the p-value first drops below 0.05?▾
Not in this fixed-horizon workflow. Repeatedly checking and stopping at the first small p-value inflates false positives. Follow the sample or horizon chosen before launch, or use a properly configured sequential method.
Does a non-significant result prove there is no effect?▾
No. It means the declared test did not cross its decision threshold at this horizon. Interpret the effect interval against the smallest effect that would matter to the business; do not treat non-significance as proof of equivalence.