A/B Test Significance Calculator

It means the gap between your two conversion rates is unlikely to be down to random chance. A significant result at 95% confidence implies there is less than a 5 percent probability of seeing a difference this large if the variants truly performed the same.

The p-value is the probability of observing a difference at least as extreme as yours if the two variants actually had identical conversion rates. A smaller p-value is stronger evidence of a real effect. The test calls a result significant when the p-value falls below 1 minus your confidence level (0.05 at 95%).

There is no single number. Smaller true differences and higher conversion rates need larger samples to detect. If your result is not significant, the usual fix is more traffic per variant. Avoid stopping the test the moment it first turns significant, since that inflates false positives.

It is a two-tailed test, which checks whether B differs from A in either direction (better or worse). Two-tailed is the safer default for most experiments. A one-tailed test only asks whether B is better, which roughly halves the p-value but is easier to misuse.

It runs a two-proportion z-test. The conversions are pooled to estimate a shared rate, that rate sets the standard error of the difference, and the z-score divides the observed difference by that error. The z-score is converted to a two-tailed p-value using the normal distribution.

No. The calculation runs entirely in your browser. Your visitor and conversion numbers are never uploaded or stored.