Online Tool Store
A/B Test Significance Calculator
Calculate statistical significance for an A/B test using a two-proportion z-test, with conversion rates, uplift, and p-value. Runs entirely in your browser.
🔒 This tool runs entirely in your browser. Your files are never uploaded to a server.
Variant A (control)
Variant B (test)
Conversion rates
Relative uplift
P-value
How the A/B test calculator works
- Enter visitor and conversion counts for variant A (control) and variant B (test).
- Choose your desired confidence level - 90%, 95%, or 99%.
- See the conversion rates, relative uplift, p-value, and a plain-language significance verdict.
FAQ
What statistical test does this use?
A two-proportion z-test with pooled variance - the standard method for comparing conversion rates between two independent groups.
What does the p-value mean?
It's the probability of seeing a difference this large (or larger) between A and B purely by chance, assuming there's actually no real difference. A smaller p-value means stronger evidence of a real effect.
What confidence level should I use?
95% is the most common standard for A/B testing. Use 99% for high-stakes decisions where false positives are costly, or 90% for quicker, lower-stakes tests.
What if my test isn't statistically significant?
It means the observed difference could plausibly be random noise. Consider running the test longer to collect more visitors and conversions before drawing a conclusion.
Does a higher confidence level make my test "more significant"?
Not exactly — a higher confidence level (99% vs. 95%) raises the bar for calling a result significant. The same data can pass at 95% but fail at 99%; the underlying data doesn't change, only the threshold for the verdict.
How we compare
| Feature | Online Tool Store | Manual statistics | A/B testing platform dashboard |
|---|---|---|---|
| Instant, no account required | ✓ | ✗ | ✗ |
| Plain-language verdict | ✓ | ✗ | ✓ |
| Works with any data source | ✓ | ✓ | ✗ |
Useful for quickly checking test results pulled from analytics tools that don't calculate significance themselves.