How to Create a Multivariate Test in VWO?

A Multivariate test (MVT) campaign is typically created to test multiple elements on a single page and find the optimal combination of those changes. To know more, refer to Multivariate testing in VWO.

Steps to Create a Multivariate Test in VWO

Prerequisite

You need to add the VWO SmartCode to the web pages you want to test. To learn how to add VWO SmartCode to your website, refer to Configuring SmartCode for your Website. 

On the main menu of your VWO dashboard, go to Testing > Multivariate > Create. The campaign is created, and it opens the Configuration tab, where you can define the details of your campaign as follows.

Declare Your Campaign URL(s)

In the URLs / Pages section, specify the URL of the webpage you want to test and click Next.

NOTE: Click the Advanced tab next to the URL field to include or exclude URLs from the campaign. Using this tab, you can also define advanced pattern-matching options to include a broad URL with wildcard entries or a specific URL. To learn about using different URL patterns, refer to Using URL Patterns in VWO.

INFO: In the case of specifying a URL pattern for the campaign, where you wish to exclude a given webpage that matches the pattern criteria, the Exclude URL option comes in handy.

For example: Let’s say you wish to run a site-wide test on https://vwo.com, but you want to exclude the Contact Us page (https://vwo.com/contact-us/) from it. Then, you need to mention the URLs in the respective fields as follows:

Included URLs: https://vwo.com*

Excluded URLs: https://vwo.com/contact-us/

Generate Combinations

In the Combinations section, you get to create variations to generate combinations by doing the following:

1. The Editor URL field displays the URL that you have specified in the Pages section by default. You can edit this URL by clicking on the Edit icon next to the URL. 
2. You can set the View mode to Desktop, Mobile, or Tablet, which will enable the editor to load the view of the webpage correspondingly.
   
3. Click Open Visual Editor. The visual editor opens for you to make changes to your webpage(s). To know more about creating variations, refer to Creating Variations Using Visual Editor in VWO.
4. The table in the Combinations page populates the control (baseline), which is the original version of the webpage and all the variations that you have created in the editor. It as well, features the number of modifications applied to each variation.
   You can also configure the traffic split among the variations and control. By default, it is split equally. You can switch it to the Auto Distribute (multi-arm bandit) mode.
5. Click Next.

Define Your Goals / Metrics

On the Goals / Metrics (for accounts enabled with Data360) page, you get to define the key performance indicators to be tracked for determining the campaign results. For example, an action such as arriving at the home page and clicking on the subscribe button in the signup form that appears. In short, it requires you to specify what action on which page to measure the conversions.

To set up goals for your test campaign, perform the following steps:

1. By default, your goal is named as Goal 1. You can rename it by clicking on the edit icon present next to the goal name.
2. From the Goal Type dropdown, select the goal type you want to work with. To know more about different goal types in VWO, refer to Different Types of Conversion Goals in VWO. 
3. In the Include URLs field, specify the operation from the dropdown, and in the field next to it, enter the URL of the webpage where you want to track the conversions. If you wish to exclude a webpage from the goal conversion tracking, use the Exclude URL option to specify the operation and the URL of that page.
   
   NOTE: This field appears based on what you have selected in the Goal Type dropdown.
4. To track another activity in your campaign, click Add Another Goal.
   
   NOTE: If you add multiple goals, you can specify one goal as Primary Goal. VWO declares a variation as a smart decision or a winner based on the primary goal. If you have multiple goals in your test, then the goal directly and exclusively impacted by the changes you make in the test must be chosen as the primary goal. By default, the first goal is considered a Primary Goal.
5. Additionally, at the bottom of the page, the Frequently used goals section is present, which shows the goals you use the most. You can either search or choose from the ones displayed. Selecting a goal from here allows you to reuse the goal code instead of configuring a new one.
   
   NOTE: The frequently used goals appear based on the domain specified for the campaign URL.

If your VWO account is enabled with Data360, you will be able to use Metrics instead of Goals. Metrics are more flexible and highly configurable. On the Metrics page, you can declare multiple metrics, out of which one needs to be primary, and the rest can be kept as secondary. The primary metric is the one based on which a variation is declared as a winner or a smart decision. It is also the one that is directly and exclusively impacted by the changes you make in the test campaign. To set up these metrics for your test campaign, perform the following steps:

1. Under the Primary metric section, click Add primary metric.
2. You can either select one of the metrics you have already created in the Data360 module or create one by clicking Create a new metric. 
3. In the Create Metric popup that appears, enter the name of the metric and select the event in the dropdown. You can choose either a system-defined event from Standard Events or a custom event from My Events. 
4. Now click on +where to specify the URL of the page/element on which you want to track the event. Specify the URL matching condition and specify the URL of the web page. You can include or exclude pages as required. You can add more locations by clicking +and where and creating a criterion for locating the pages/elements. To delete a location criterion or a URL, click on the respective Delete icons.
5. You can configure how you want the metric to measure the conversion using the following options:

• Unique Visitors - Each visitor, upon triggering the event, can be counted as converted only once, irrespective of the number of times the event is triggered by the same visitor. E.g. You can use this option to track if a visitor has clicked on a link.
• Event Totals - The event is counted as converted every single time a visitor triggers it. This way, it also populates multiple conversions made by the same visitor. E.g. You can use this option to track the total clicks on a CTA button, such as Add to Cart.

NOTE: The Event Totals option is not included with the VWO Data360 Growth plan.

• Value of an event property - Calculates the conversion of the event based on the property that you define. This can be used for those events that involve properties with numerical values, such as tracking revenue and time spent on a page. To apply this, you must define the event property and its corresponding value type (Sum, Average, Max, Min, First, and Last).

NOTE: In case of tracking revenue, select the Format this metric as currency checkbox. This populates the values in the currency (as configured in your workspace-level campaign settings. Read more) in your reports.

6. To save the metric and apply it to your future campaigns, click on the Add this metric to Data360 for future use checkbox at the bottom left and click Create. 

INFO: The saved metrics can be managed from Data360 > Metrics.

7. Select the metric to apply to the campaign.
8. To add any additional metrics, click Add secondary metric under the Secondary metrics section and repeat the steps of adding the primary metric.

Estimate the Duration of Your Campaign

Upon creating a combination, the Duration section is enabled. This allows you to estimate the duration of your MVT campaign. Refer to this article to learn more about it.

Configure Your Integrations

On the Integrations page, you can apply to your campaign any third-party application integrations that you have already enabled in your account. If you haven’t yet integrated with any third-party application, click on Browse more integrations…, select the respective third-party application integration tile, and click on the Enable Now button to enable the corresponding integration.

More Configurations

Now your campaign is ready with all the fundamental requirements to be started. However, a typical campaign generally tends to be applied with certain ancillary configurations. This includes visitor segmentation, triggering of the campaign, scheduling the campaign, and many more. All of those ancillary configurations can be found under the More configurations dropdown as follows:

Reviewing Your Campaign

Once you have supplied all the required details for running the campaign, the Review tab on the left pane gets enabled and allows you to review your campaign configurations, preview your campaign and view the screenshots of your variations. It features the following sections:

• Configuration summary - Lists all the configurations that you have applied to your campaign. It includes all the essential and ancillary settings that were applied to your campaign. You can quickly run through the summary and make changes if necessary.  Apart from that, if you’ve entered any URL that involves an unregistered deployment (domain), that will also be listed at the top of the page. You can click the Register link to register the deployment in your VWO account. Follow this article to learn more about Websites and Apps in VWO.
• Live Preview - Allows you to preview your campaign combinations.
• Variation Screenshots - Allows you to view the screenshots of the control and the combinations that you have created. To view the screenshots, hover on the respective screenshot thumbnail and click View Screenshot. You can view the screenshots for different browsers by selecting the required options from the Select Browsers dropdown. By default, the screenshots are grouped based on browsers. You can switch the grouping to be based on combinations by selecting the appropriate option from the Group by dropdown.

NOTE: If you’re viewing the screenshots for the first time, they may take a few seconds to generate, depending on the server load.

Managing Your Campaign

The following options are available on the left pane on the campaign creation page, allowing you to manage your campaign:

	Click on this to view the timeline of the events and changes that were applied to your campaign
	Click on this to clone your campaign
	Click on this to flush your campaign data. TIP: We recommend that you do not flush the test data and run the same test again. Rather clone the test first, flush data from it, and implement the changes to the cloned test. To learn more about flushing your campaign data refer to this article.
	Click on this to archive your campaign. Archiving a test removes it from the dashboard, but you can still access it from the archived list. You can do this by applying the Archive filter on the A/B test dashboard
	Click on this to delete the campaign. You can retrieve a deleted campaign by clicking on the retrieve link on the left pane and unarchive the same to view it in the campaigns list
	Click on this to email the link to the campaign to your team
	Click on this to print the campaign configuration summary
	Click on this to download the campaign report data as a CSV file

Post-requisite

To ensure that the changes you made to the variations are in place and work as expected for your website visitors, preview the combinations of your variations before actually starting a campaign. Go to Review > Live Preview in your campaign.

The Combinations dropdown features the list of sections, from each of which you can choose the control or a variation to create a combination. 

The  Preview URL features the URL that you've mentioned in the Pages section. You can also override it with any specific URL where you have applied the variations.

Post configuring, click on the Preview in a new tab button.