Interpreting VWO Personalize Campaign Reports

Overview of Personalize Reports

A VWO Personalize campaign report consists of the statistical data of the campaign outcomes such as Current Traffic Split, Unique Conversions/Visitors, Expected Conversion Rate, Expected Improvement, and Probability of Better or Equivalent numbers. This data is presented as tables and graphs. VWO Personalize campaign reports provide valuable insights into the performance of your personalization efforts. The reports allow you to analyze the effectiveness of different experiences and make data-driven decisions to optimize your campaigns. The Holdback feature significantly impacts the data presented. You'll see different reports depending on whether you have enabled or disabled Holdback. Within each mode, you can view your data grouped by Experience for an aggregate view or by Target for a more granular breakdown by segment.

Access Personalize Reports

To access your Personalize campaign report, go to Personalize, and select the campaign for which you want to view the report. You can view the report for your campaign under the Reports tab.

It features three sections - Reports that highlights the outcomes of your Personalize campaign, Graphs that present the data graphically, and Filters that can be applied to the results for targeted study.

Filter Personalize Reports

As an initial step, you must set up your filter to view the corresponding report. You can set up the date range, segment of visitors, and visitor dimensions to fetch the data accordingly.

For more information on how to get filtered reports based on the date range, visitor dimensions, and other visitor attributes, see Filter Your Report Data.

Understand the Report Table

VWO Personalize campaign reports offer four distinct structures based on the combination of the Holdback setting configuration and Group By selection. You can use the Group By dropdown to group the report data by:

• Experiences
  Use this view to get a high-level overview of how each experience is performing across all your target segments. This is helpful for identifying which experiences are generally performing well or poorly.
  For example, if you have three experiences (A, B, and C) running across two targets (Target 1 and Target 2), the Experiences view will show you the combined performance of Experience A across both Target 1 and Target 2, the combined performance of Experience B across both targets, and so on. This aggregated view is helpful for quickly identifying winning or losing experiences without getting bogged down in segment-specific details.
• Targets
  Use this view to drill down into the performance of each experience within a specific target segment. This is helpful for understanding how different variations resonate with different audiences and for identifying opportunities for optimization within each segment.
  Using the same example as above, the Targets view will show you the performance of Experience A, B, and C specifically within Target 1, and then the performance of those same experiences specifically within Target 2. This segmented view is crucial for understanding how different variations perform with different audiences and for tailoring your personalization strategy accordingly.

This flexibility allows you to analyze your campaign performance from various perspectives.

Here's a breakdown of each report structure and its parameters:

Holdback: Disabled and Group By: Experiences

When Holdback is off, the report compares the performance of individual experiences within your campaign. As the report is grouped by Experiences, the report provides an aggregate overview of how each experience performs across all targets.

• Purpose: Compare the performance of different experiences across all target segments.
• Focus: Identify the best-performing experience overall.
• Data Displayed:
  • Experiences: Lists each experience in the campaign.
  • Current Traffic Split: Percentage of traffic allocated to each experience.
  • Total Value (Unique Conversions/Visitors): Total value, unique conversions, and visitors for each experience.
• Visualizations: Date Range Graph and Traffic Split.
• Example: Compare the performance of Experience 1, Experience 2, and Experience 3 across all target segments combined.
  

Holdback: Disabled and Group By: Targets

When Holdback is off, the report focuses on comparing the performance of individual experiences within your campaign. As the report is grouped by Targets, this view provides a more granular breakdown of campaign performance by target segment.

• Purpose: Compare the performance of different experiences within each target segment.
• Focus: Understand how different variations resonate with specific audiences.
• Data Displayed:
  • Targets: Lists each target segment.
  • Experiences: Lists each experience within each target.
  • Current Traffic Split: Percentage of traffic allocated to each target.
  • Total Value (Unique Conversions/Visitors): Total value, unique conversions, and total visitors for each target and each experience within the target.
• Visualizations: Date Range Graph and Traffic Split.
• Example: Compare the performance of Experience 1, Experience 2, and Experience 3 specifically within Target 1 and then separately within Target 2.
  

Holdback: Enabled and Group By: Experiences

When Holdback is on, VWO creates a control group, providing a baseline against which to measure the impact of your personalized experiences. As the report is grouped by Experiences, VWO groups all your personalized experiences together for comparison. This allows you to assess the overall impact of your personalization strategy by measuring the combined performance of all experiences against the baseline of the Holdback group. In addition, this report shows the expected value per visitor for each experience and the holdback group, the overall expected improvement in the primary metric compared to the holdback group, and the probability that the experiences will outperform the holdback group. These additional metrics provide statistical significance to your results, helping you confidently determine whether personalization drives positive change.

• Purpose: Compare the combined performance of all experiences against the holdback (control) group.
• Focus: Measure the overall impact of personalization.
• Data Displayed:
  • Holdback: Represents the control group.
  • Experiences: Groups all experiences together.
  • Current Traffic Split: Percentage of traffic allocated to the holdback group, the combined experiences, and the individual experiences.
  • Unique Conversions/Visitors: Total value, unique conversions, and visitors for the holdback group, the combined experiences, and the individual experiences.
  • Expected Conversion Rate: Expected conversion rate for combined experiences, each experience, and the holdback group.
  • Expected Improvement: Expected improvement in the primary metric for the combined experiences compared to the holdback group.
  • Probability of Better: Probability that the combined experiences will outperform the holdback group.
• Visualizations: Date Range Graph, Traffic Split, and Expected Improvement Graph.
• Example: Compare the combined performance of Experience 1, Experience 2, and Experience 3 against the holdback group to determine the overall impact of personalization.
  

Holdback: Enabled and Group By: Targets

When Holdback is on, VWO creates a control group, providing a baseline against which to measure the impact of your personalized experiences. As the report is grouped by Targets, VWO provides a granular breakdown of your campaign performance by target segment, including a Holdback group for each target. This allows you to measure the impact of personalization within specific audience segments and understand how different variations resonate with different audiences.

• Purpose: Compare the performance of different experiences within each target segment, including the holdback group for that target.
• Focus: Measure the impact of personalization within specific audience segments.
• Data Displayed:
  • Targets: Lists each target segment.
  • Holdback: Represents the control group within each target.
  • Experiences: Lists each experience within each target.
  • Current Traffic Split: Percentage of traffic allocated to each target.
  • Unique Conversions/Visitors: Total value, unique conversions, and visitors for the holdback group, each target, and experience within each target.
  • Expected Conversion Rate: Expected conversion rate for each experience and the holdback group within each target.
  • Expected Improvement: Expected improvement in the primary metric for each experience compared to the holdback group within each target.
• Visualizations: Date Range Graph, Traffic Split, and Expected Improvement Graph.
• Example: Compare the performance of Experience 1, Experience 2, and Experience 3 against the holdback group, specifically within Target 1 and then separately within Target 2. This allows you to see the impact of personalization on each segment individually.
  

The report table displays the data collected for each experience in the campaign. You can disable the experiences or preview the experiences to verify your campaign settings using Disable Experience and Preview Experience options, respectively from the vertical ellipsis icon (⋮) present at the right end of the table row.

You can choose to filter the experiences and sort the data based on the parameters. Just click on the required parameter on the header row to sort the data in the ascending/descending order.

Understand the Graph View

The graphs provide information on the conversion rate, conversions, and visitors over the configured period of time. To view the graph for a different period of time, visitor segment, or visitor dimension, modify the report filters appropriately.

The graph section contains the following two parts:

• Date Range - This features the conversion rates, conversions, visitors, and revenue per visitor data.
• Traffic Split - This features the split of the overall traffic for each experience.
• Expected Improvement Graph - This is generated only if Holdback is enabled in the campaign. It represents a specific variation's expected improvement over the baseline.

Interpreting the Date Range Graph

Using the Date Range graph you can read the conversion rates, conversions, visitors, and the revenue per visitor by configuring them in the dropdown at the top-left of the graph.

In case of selecting the Conversion Rate (%) option, the Show ranges option appears at the top-right of the graph. The Show ranges option displays the median, the best case and the worst case values of the expected conversion rate corresponding to the experience. Through this, you can also figure out the uncertainty overlap an experience may share with the others.

The example in the image below shows the comparison among the experiences (E1, E2, E3, and E4). Each version is highlighted in a different colour that is indicated at the bottom of the graph. Initially, all the versions are at mark zero. As soon as the inflow of visitor traffic starts, data collection begins, and the conversion rate is tracked. Based on the data collected over time, the change in graph pattern could be seen.

In the above graph, the x-axis represents the dates, and the y-axis represents the conversion rate.

Interpreting the Traffic Split

The overall traffic to your campaign is represented by the donut that is divided into different portions based on the number of experiences. The size of each portion represents the percentage of the amount of traffic that receives that experience.

For more information on how to interpret the graphs, see Leverage Graphs to Analyze Your Campaign Report.

Need more help?

For further assistance or more information, contact VWO Support.