Multivariate Testing (MVT) is the testing technique wherein multiple elements on a single page are changed to determine the optimal combination of those changes. The objective of MVT is to find out which combination of elements works best for your audience. In VWO, you would typically run an MVT to test the visual elements, CTA buttons, form fields, and texts on your webpage.
MVT helps you to:
- Optimize smaller changes on your webpage
- Determine the impact of each element on your webpage
- Test multiple changes in a single test
- Reuse the data about the impact of a certain element’s design as it can be applied to future campaigns, even if the element's context has changed.
While A/B testing and Split URL testing allow you to compare two or more versions of a webpage by splitting traffic equally among them, MVT allows changes on more than one element on a page and helps you find the best converting combination. Also, the MVT test is more complex as it compares several elements to find out those elements which have the highest impact on your website. This helps you determine which combination of elements has the highest conversion rate.
The total number of variations in an MVT is calculated as:
[# of Variations on Element A] X [# of Variations on Element B] … = [Total # of Variations]
For example, you might want to test two versions, each of a heading and an image on a webpage. Using MVT, you create one variation for the heading and one for the image. To test all the versions now, create a combination of all the variations (2 X 2 = 4), as shown here:
- Version 1: Heading 1 + Image A
- Version 2: Heading 2 + Image A
- Version 3: Heading 1 + Image B
- Version 4: Heading 2 + Image B
PRO-TIP
1. To get the best out of MVT testing, it's recommended to use it only
when your website receives high traffic. Based on this, you can calculate
the estimated time that your MVT test will take to produce results.
To do this,use the VWO’s Test Duration Calculator.
NOTE: In the event that your MVT test takes too long to declare results,
we recommend breaking down your MVT test into various A / B tests. This
helps to reduce the number of combinations required for testing.
2. A Multivariate test is best-suited for situations wherein you have a
hypothesis of multiple ways to implement a variation, but you are unable
to decide which combination should be tested.
When Not to Use Multivariate Testing?
It would be best if you do not use Multivariate Testing when:
- The volume of visitors landing on your website is less. This is because in an MVT test, the combinations to test increase, and you must have a fairly big amount of traffic and conversions
- You have less time to test because more element combinations require a longer to test
When Should You Choose Multivariate over A/B?
If your objective is to test the smaller elements instead of completely revamping your web page to figure out the element that makes the most impact on the conversion rate, you must use Multivariate Testing. Also, it would help if you were sure that there is enough traffic and conversions on your website.