What is Multivariate Testing?

Multivariate Testing (MVT) is the testing technique wherein multiple elements on a single page are changed to determine the optimal combination of those changes. The objective of MVT is to find out which combination of elements works best for your audience. In VWO, you would typically run an MVT to test the visual elements, CTA buttons, form fields, and texts on your webpage.

MVT helps you to:

• Optimize smaller changes on your webpage
• Determine the impact of each element on your webpage
• Test multiple changes in a single test
• Reuse the data about the impact of a certain element’s design as it can be applied to future campaigns, even if the context of the element has changed.

While A/B testing and Split URL testing allows you to compare two or more versions of a webpage by splitting traffic equally among them, MVT allows changes on more than one element on a page and helps you find the best converting combination. Also, the MVT test is more complex as it compares a number of elements to find out those elements which have the highest impact on your website. This helps you determine which combination of elements has the highest conversion rate. 

The total number of variations in an MVT is calculated as:

[# of Variations on Element A] X [# of Variations on Element B] … = [Total # of Variations]

For example, you might want to test two versions each of a heading and an image on a webpage. Using MVT, you create one variation for the heading and one for the image. To test all the versions now, create a combination of all the variations (2 X 2 = 4), as shown here:

• Version 1: Heading 1 + Image A
• Version 2: Heading 2 + Image A
• Version 3: Heading 1 + Image B
• Version 4: Heading 2 + Image B

PRO-TIP
1. To get the best out of MVT testing, it's recommended to use it only when 
your website receives high traffic. Based on this, you can calculate the 
estimated time that your MVT test will take to produce results. To do this,
use the VWO’s Test Duration Calculator. 
NOTE: In the event that your MVT test takes too long to declare results, 
we recommend breaking down your MVT test into various A / B tests. This 
helps to reduce the number of combinations required for testing.

2. A Multivariate test is best-suited for situations wherein you have a 
hypothesis of multiple ways to implement a variation, but you are unable 
to decide which combination should be tested. 

When Not to Use Multivariate Testing?

You should not use Multivariate Testing when:

• The volume of visitors landing on your website is less. This is because in an MVT test the combinations to test increase and it is important that you have a fairly big amount of traffic and conversions
• You have less time to test because more element combinations require longer to test
  

When Should You Choose Multivariate over A/B?

If your objective is to test the smaller elements instead of completely revamping your webpage with the intent of figuring out the element which makes the most impact on the conversion rate, you must use Multivariate Testing. Also, you need to be sure that there is enough traffic and conversions on your website.