Meta-Analysis Calculator

This calculator uses a fixed-effects meta-analysis with inverse-variance weighting, which is the standard approach for combining A/B test results. The core formulas are: Weight for study i: w_i = 1 / SE_i² Pooled effect: θ = Σ(w_i × θ_i) / Σ(w_i) Pooled standard error: SE(θ) = √(1 / Σ(w_i)) Where: - θ_i is the observed effect (lift %) for study i - SE_i is the standard error for study i - w_i is the inverse-variance weight The 95% confidence interval is: θ ± 1.96 × SE(θ) Each study must include its actual standard error on the same effect scale as the reported effect. If you have a two-sided 95% confidence interval, derive SE as (upper - lower) / (2 × 1.96). Sample size and lift alone are not enough to reconstruct a standard error. This fixed-effect model assumes every included study estimates one common true effect. It is only appropriate when the outcome definition, effect scale, treatment contrast, analysis unit, and target population are comparable. A low I² value does not prove comparability, especially with only a few studies. If effects plausibly vary by brand, page, device, or audience, do not interpret this pooled estimate as universal; use a justified random-effects or hierarchical analysis outside this calculator. Heterogeneity is assessed using Cochran's Q statistic: Q = Σ w_i × (θ_i - θ)² Q follows a chi-squared distribution with k-1 degrees of freedom under the null hypothesis of homogeneity. The I² statistic quantifies the proportion of total variation due to true heterogeneity rather than sampling error: I² = max(0, (Q - df) / Q × 100) I² interpretation: 0–25% low, 25–75% moderate, 75%+ high heterogeneity.