|
In this article, you’ll learn the following: |
Overview
The moment you start your campaign, the VWO SmartStats engine kicks in to collect data and determine when to display results. It monitors visitor traffic and conversion trends on your website. To ensure reliable results, SmartStats requires a minimum of 500 visitors and at least one conversion in the respective variations.
Once this threshold is met, SmartStats leverages your statistical settings to estimate the time needed for the campaign to reach your desired improvement. This is when you'll see conversions and estimated timelines for the primary metric. Now is the time for you to roll up your sleeves to mine insights from your report data.
Tailor the Look of Your Report
To read the report more effectively, you can configure its design to suit your preference/requirement. The View Settings option allows you to configure your report according to your preferred appearance.
Click on the gear icon at the top right of the report table to access the View Settings popup. It features two sections: Layout options and Table options.
Layout Options
By default, VWO displays the report table before the graph. If you’re a marketing manager who prefers a visual representation of data, you can choose the Graphs first option to prioritize graphs over tables for quick identification of trends and patterns. Also, you can choose which graph to be displayed by default by selecting the appropriate option under Default Graph.
Table Options
You generally get to see the median values of the conversion and the improvement rates.
As an e-commerce store owner, you might be interested in understanding the potential range of conversion rates. To keep track of these parameters' ranges, you can select the Expected conversion rate and Expected improvement under Bayesian ranges.
Besides, if you don’t need the Total row or wish to hide the row for a disabled variation, deselect the Show 'total' row and Show rows for disabled variations options.
Once you’re done configuring, click Save. To rest to default, click on the Restore Defaults button.
Report Table
The report table presents the numerical values of the performances of the variations for the given test metric through the following parameters:
For binary metrics
| Unique conversions/ Visitors |
Shows the number of unique conversions (Successes) that happened with the number of visitors without counting repeated conversions from the same visitor |
| Expected conversion rate | Shows the statistically expected conversion rate (Success rate) for all variations, indicating the anticipated conversion rate in the future based on collected data |
For continuous metrics
|
Total value Unique conversions/ Visitors |
Shows the sum of the metric value for each visitor, the number of unique conversions (Successes) that happened with the number of visitors without counting repeated conversions from the same visitor |
| Expected value per visitor | Shows the statistically expected metric value per visitor for all variations, indicating the anticipated expected metric rate in the future based on collected data |
Common Parameters
| Expected Improvement | Shows the statistically expected improvement in a variation compared to the baseline, with intervals indicating the potential range of improvement that narrows down as more data is collected |
| Probability of Better |
Shows the probability that a given variation provides an improvement over the baseline. Once this probability exceeds the defined threshold, a decision (whether to roll out a variation or exclude a variation from the campaign) is made. This value serves as the primary indicator of statistical significance in your test. We highly recommend considering a variation to be better and worse than the baseline only if this value crosses the desired thresholds. |
Take Actions on Your Variations
For Baseline
| Disable | To turn off the variation and stop the data collection for the same |
| Preview | To preview the variation at any required time |
For a Variation
| Use this as the baseline | To keep the variation as the baseline to compare the performances of the other variations |
| Disable | To turn off the variation and stop the data collection for the same |
| Preview | To preview the variation at any required time |
| Roll out | To roll out the variation on your website following the decisions that you arrive at based on the statistical report |
| Edit |
To make modifications to the variation. ATTENTION: VWO strongly advises against making changes to a variation after the start of a campaign, as it would skew the results. If needed, you can clone the campaign and make the desired modifications. |
Arrange Your Report Data
Sort
To compare the data within the variations, you can sort them in ascending or descending order for any given parameter at a time. Just click on the required parameter on the header row to toggle between the orders.
For example, you want to quickly identify the variation with the highest conversion rate. Sorting the Unique Conversions/Visitors column in descending order instantly reveals the top performer.
Filter
For a focused study, you can filter the variations. Click Variations on the header row, select the variations you want to focus on, and click Apply.
For instance, you're specifically interested in comparing two variations. Filtering the report to display only these two variations provides a concentrated view, facilitating side-by-side analysis.
Campaign Status Features
The report table comes with the following features along with the campaign stats:
| This features the Experiment Vitals, such as the minimum run time for a campaign, conversion tracking, and data tracking, that are required for your campaign to render the most reliable results. When all the vital parameters are set right, the icon glows green. Else, it glows red and shows the number of vitals that are breached. You can read the alert and check the parameter that has lapsed the requirement | |
|
This displays the Statistical Configuration set for your campaign as follows:
VWO pre-configures default statistical settings for all new campaigns, applying a correction factor to the expected improvement calculation. We recommend that you be prudent while adjusting these statistical parameters to meet your experimentation goals |
|
| This displays the Learnings that you have deduced from the report. You can create a Learning by clicking on this icon and selecting the Add new learning option | |
| This is the Refresh option that, upon clicking, updates the data to the current state | |
|
This is the Notifications icon, which, when clicked, displays the Notifications popup, which features two tabs:
|