Why Am I Seeing Results So Early?
SmartStats is designed to give you intelligent answers quickly to make smart business decisions. Compared to traditional frequentist-based tools, SmartStats usually provides answers in less time unless your base conversion rate is above 15%.
It’s important to note that we gain this speedup through several trade-offs than the traditional frequentist methods. Most traditional statistical methods were designed for publishing scientific papers and to determine whether an effect is real. The stakes are different on the web. Either variation A or variation B is deployed; the question a company asks is “Which option will make us more money?”
You should run your test for an integer number of weeks, and the VWO app warns you if you stop the test after five or eight days. As the test (which contains one Monday for every Saturday) is supposed to be representative of the future, it’s usually a best practice to make sure that the test has the same number of Mondays and Saturdays in the test period.
What Input Do I Need to Enter for SmartStats?
To adopt SmartStats, you will require your website’s average daily visitor data and the current conversion rate. To find out these data, you can run a Conversion Tracking test in VWO. Alternatively, you can gather the data from Google Analytics, Mixpanel, Kissmetrics, or any other analytics tool.
For the lift in the conversion rate, you should choose a realistic value. Usually, a 15–20% change is worth caring about. For instance, if the current conversion rate of your page is 6%, then you may be interested only if the conversion rate of the variation goes up by 20%. (A 20% increase will boost your conversion rate from 6% to 7%.). Click here to learn more about the SmartStats way of finding your winning variation.