|
In this article, you’ll learn: |
NOTE: We've implemented a fresh experience for reports improved with statistical enhancements. It is being rolled out in phases. If you're experiencing the latest version, refer to this article.
Deriving meaningful insights from your test campaigns and applying them to improve your website conversion rate is your test campaigns' ultimate objective. The test campaign reports help you validate the hypothesis you test and derive a better understanding of your website's visitor behavior.
To access your test report, select the campaign for which you want to view the report and go to the REPORT tab.
To view the report corresponding to a goal, select one from the GOALS panel on the right. Additionally, you can also compare the data of different goals using the COMPARE GOALS option.
Viewing Report for a Date Range
Date filter enables you to filter data for the specified date range. To specify a date range, you can either use the existing options or can specify a custom date range using the calendar control.
Viewing Report by Visitor Segments
To filter the report by visitor segments, click the All Visitors drop-down menu and select the segment you want to apply. You can use the Segment Gallery (default VWO segments) or filter the report per the segment you want to apply.
To learn how to create a custom segment, click here.
Warnings and Notifications
The notification box displays the status of your campaign and any warnings you must be aware of.
- The notification section displays the campaign's status and any other issues with data collection in a variation or with conversion tracking. You can also view the Winner and Smart Decision notifications here.
- Warnings are displayed if errors were made when setting up the campaign, which can impact your report's data integrity. For example, if you modify a test after it starts running.
- Insights you derive from your test report, or even fresh ideas you may want to test.
To know more about VWO warnings, refer to Warnings in VWO.
Customizing the Report View
You can customize the layout of your report using the View Settings option. To access this option, click on the gear icon present next to the stats table.
- Layout Options: From this section, you can customize the view of your table and graph reports.
- Table Options: From this section, you can add or remove columns to the report table, customize table rows to display, and update the data you want to view in the report. You can also move the columns to change how these should appear in the table.
For more details, refer to View Settings.
Understanding the Reports Table
The report table displays the data collected for each variation included in the campaign. By default, the control is set as the base for comparing that data. You can change the base to any other variation for calculations using the Use this as Baseline option and can disable or preview the variations using Disable variation and Preview variation options from the vertical ellipsis icon (⋮) present at the right end of the table row.
- Expected conversion rate/Expected revenue per visitor: This is the median conversion rate you can expect from the variation. The "best case" and "worst case" conversion rates represent the 99% credible interval where the conversion rate is likely to be contained.
- Improvement/Revenue improvement: This is the median improvement you can expect over the baseline if you implement the variation. The "best case" and "worst case" values represent the 99% credible interval where improvement is likely to be contained.
- Probability to beat control: The probability of a variation to perform better than the baseline. By default, control is the baseline.
- Probability to be best: This depicts the number of times a variation performed better than all other variations, including control.
- Expected revenue per conversion: This is the median revenue per visitor you can expect from the variation. The "best case" and "worst case" revenue per visitor values represent the 99% credible interval where they are most likely to be contained.
- Conversions/Visitors: The ratio of the number of actual goal conversions to the total number of page visitors.
- Absolute potential loss: This depicts the differences between all the sample sets where variation is not a winner. Absolute Potential Loss finds out how much you stand to lose in a scenario where the variation in question always loses to another variation.
Understanding the Graph View
The graph provides information on the conversion rate, conversions, visitors, revenue per visitor, and revenue. You can view the graph in either of the following ways:
- Date Range
- Box Plot
- Probability Density
Date Range
In the Date Range view, you can view the Conversion Rate, Conversions, Visitors, and Revenue per visitors' data over a period of time. You can choose a custom date range using the Date Filter. For more information on Date Filter, refer to the View Report for a Date Range section.
Interpreting the Change in Conversions
The Graph below shows the comparison between the Control(C) and Variation(V1). The orange line denotes the Control, whereas the blue line denotes the Variation. Initially, both the Control and Variation are at mark zero. As soon as the inflow of visitor traffic starts, data collection begins, and the conversions are tracked. Based on the data collected over time, the change in graph pattern could be seen.
In the above graph, the x-axis represents the dates, and the y-axis represents the conversions. To identify the change between the two, let’s look at the graph pattern of June 21. It clearly shows the number of conversions in the Variation is well beyond the Control. Likewise, you can interpret the graph pattern for all other date ranges.
You can also turn on ranges and see how the conversion rate range for variations have evolved over time. Ideally, the ranges become smaller over time, and that should reflect in the graph. If the ranges go wide, it indicates fluctuations in the data. If the ranges overlap, it signifies uncertainty about which variation is performing better. Read more about uncertainty overlap in the Box Plot section.
Box Plot
In the Box Plot view, you can view Revenue, Revenue Per Visitor, Conversions, Conversion Rate, and Visitors as box plots to see any uncertainty overlap amongst Variations and Baseline.
Interpreting the Change in Box Plot
In this graph, the box plots of Control and Variations are shown along with the uncertainty overlap. The orange is the box plot of Control, and the other in blue is the box plot of Variation. Based on the data collected, the Best Case, Most Likely, and Worst Case are derived. Here, the Most Likely value represents the median value of the conversion rate. The Best Case and Worst Case values represent extreme cases. In the probability distribution curve below, you’ll see that Best Case and Worst Case scenarios are way less probable than the Most Likely scenario. The more the data, the higher the probability of conversion rate near the Most Likely(median) value. The area above the uncertainty overlap indicates that the conversions are much higher in the Variation compared to the Control.
Probability Density
The probability distribution curve tells us the probability of getting a conversion rate. In VWO graphs, the x-axis represents the conversion rate, and the y-axis represents the probability. The top of the curve is the conversion rate with the highest probability of getting expected results, and the range of the graph represents the 99-percentile conversion rate range.
Interpreting the Change in Probability Density curves
Our initial guess about the conversion rate of your website is anything between 0% and 100%, this indicates the probability of any conversion number is the same. For example, the probability of the conversion rate being 4% is the same as that of 32%, and so on. When you start collecting data, we use it to update our initial guess. The updated guess is what you see in the reports as Conversion Rate. With each data point, you come closer to the actual conversion rate. If you plot the Conversion Rate against its updated probability, you get a graph as above.
In the graph, the probability density of Control and Variations are shown along with the uncertainty overlap. The orange curve represents the Control, and the other one in blue represents the curve for Variation. The median value of Variation is 2.16%, whereas, for the control, it is 0.86%. This clearly indicates that the probability of conversion is higher in Variation. The more the data, the higher will be the probability of conversion rate to be near the median value.