Statistical significance is a mathematical formula used to predict the probability that one version is better than the other. Theme Scientist includes statistical significance calculations for every A/B test.
Based on conversion rate, this percentage is the probability that one variant will outperform the other long term. The significance will start after each variation has at least one conversion and 50 sessions, but we recommend running the test for at least two weeks, or until each variation has at least twenty conversions, to minimize randomness.
While 90% significance is good, a 95%+ significance level is commonly agreed upon as the minimum number to declare a clear winner.