Skip to main content
Free Calculator

Sample Ratio Mismatch (SRM) Calculator

Check whether your A/B test traffic split matches the intended allocation. Detect assignment bugs, bot filtering issues, and other data quality problems before they bias your results.

Presets:

Results

No SRM — Traffic Split Looks Good

p-value threshold: 0.01

p-value

0.1594

Chi-Squared Statistic

1.9802

Per-Group Deviation

GroupExpectedObservedDeviation
Control10,10010,000-0.99%
Variant10,10010,200+0.99%

Methodology

This calculator uses the chi-squared goodness-of-fit test to determine whether the observed traffic split in your experiment matches the expected allocation. The test statistic is calculated as: χ² = Σ (O_i - E_i)² / E_i Where: - O_i is the observed count for group i - E_i is the expected count for group i, based on the intended ratio and total traffic - The sum is taken over all groups The expected count for each group is: E_i = (ratio_i / Σ ratios) × total_observed The p-value is computed from the chi-squared distribution with df = k - 1 degrees of freedom, where k is the number of groups. A p-value below 0.01 indicates a statistically significant sample ratio mismatch. This threshold is stricter than the typical 0.05 used in hypothesis testing because SRM is a data quality diagnostic — false positives here lead to unnecessary experiment restarts, while false negatives allow biased results to go undetected.

Frequently Asked Questions

What is a sample ratio mismatch (SRM)?▾
A sample ratio mismatch occurs when the actual traffic split in an A/B test does not match the intended allocation. For example, if you configured a 50/50 split but observe 52% in control and 48% in the variant, there may be a systematic issue causing uneven assignment. SRM is detected using a chi-squared goodness-of-fit test, and it signals that your experiment results may be unreliable.
Why does SRM matter for A/B testing?▾
SRM is a critical data quality check. When traffic is not split as intended, it often means something is systematically different between your groups — for example, certain users may be filtered out of one variation. This breaks the fundamental assumption of random assignment and can lead to biased results, making your lift estimates and p-values unreliable.
What are common causes of sample ratio mismatch?▾
Common causes include: bot filtering that disproportionately affects one variation, redirect-based implementations that lose users, browser caching serving stale assignments, bucketing bugs in the assignment logic, interaction effects with other concurrent experiments, and differences in page load speed between variations causing higher bounce in one group.
What p-value threshold should I use for SRM detection?▾
The standard threshold for SRM detection is p < 0.01, which is stricter than the typical p < 0.05 used for experiment results. This lower threshold is appropriate because SRM is a data quality check — you want to be very confident there is a real mismatch before invalidating experiment results. Some teams use p < 0.001 for even more conservative detection.
How do I fix a sample ratio mismatch?▾
First, identify the root cause by examining your assignment logic, checking for bot filtering differences, reviewing redirect implementations, and looking at timing of the mismatch. Common fixes include: ensuring consistent assignment before any user-facing changes, applying bot filters equally across variations, fixing caching headers, and verifying your randomization algorithm. You should restart the experiment after fixing the underlying issue rather than continuing with biased data.

Related Calculators

A/B Test Significance CalculatorChi-Squared Test CalculatorA/B Test Sample Size CalculatorA/B Test Duration Calculator

Updated for 2026. Built by GrowthLayer.