Skip to main content
Free Calculator

Chi-Squared Test Calculator

Test whether conversion rates differ across multiple A/B test variations using the chi-squared test of independence. Supports 2 to 10 groups.

Results

Chi-Squared Statistic

2.0759

Degrees of Freedom

1

p-value

0.1496

Significant?

No (p ≥ 0.05)

Cramer's V (effect size)

0.0322

Small effect

Observed vs Expected Counts

VariationConv. RateObs. Conv.Exp. Conv.Obs. Non-Conv.Exp. Non-Conv.
Control5.00%5057.5950942.5
Variant B6.50%6557.5935942.5

Methodology

This calculator uses the Pearson chi-squared test of independence applied to a 2 x N contingency table, where rows represent outcomes (converted, not converted) and columns represent variations. For each cell in the contingency table, the expected count under the null hypothesis of no association is: E_ij = (row_total_i * column_total_j) / grand_total The chi-squared statistic is: X^2 = sum of (O_ij - E_ij)^2 / E_ij Where O_ij is the observed count and E_ij is the expected count for each cell. The degrees of freedom for a 2 x N table is (2-1) * (N-1) = N-1, where N is the number of variations. The p-value is computed from the chi-squared CDF using the regularized incomplete gamma function. A p-value below 0.05 indicates that the observed differences in conversion rates are statistically significant at the 95% confidence level. Cramer's V is the effect size, computed as sqrt(X^2 / (n * min(r-1, c-1))). For a 2-row table, this simplifies to sqrt(X^2 / n). Values range from 0 (no association) to 1 (perfect association).

Frequently Asked Questions

When should I use a chi-squared test instead of a z-test?
Use a chi-squared test when you have more than two variations (e.g., A/B/C/D tests). A standard z-test only compares two groups at a time. The chi-squared test evaluates whether conversion rates differ across all groups simultaneously, telling you whether at least one variation performs differently from the others.
What is the chi-squared statistic?
The chi-squared statistic measures the overall discrepancy between observed and expected counts in a contingency table. Larger values indicate greater deviation from what you would expect if all variations had the same conversion rate. It is calculated by summing (observed - expected)^2 / expected across all cells.
How do I interpret the p-value from a chi-squared test?
The p-value is the probability of seeing results as extreme as yours (or more extreme) if there were truly no difference between variations. A p-value below 0.05 is typically considered statistically significant, meaning it is unlikely the observed differences are due to random chance alone.
What is Cramer's V and how do I interpret it?
Cramer's V is an effect size measure for chi-squared tests, ranging from 0 to 1. A value near 0 means the variations have almost no practical difference, while a value near 1 means a very strong association. For A/B tests with 2 rows, V below 0.1 is small, 0.1-0.3 is medium, and above 0.3 is large.
Can I use the chi-squared test with only two variations?
Yes. With two variations the chi-squared test is mathematically equivalent to a two-tailed z-test of proportions. However, if you only have two variations and want a directional result (which one is better), a z-test or Bayesian approach is more informative since chi-squared only tells you that a difference exists, not its direction.

Related Calculators

Updated for 2026. Built by GrowthLayer.