To ensure that a visitor on your website who is part of one test does not become part of another test from the same group, you can make your tests mutually exclusive to one another. This means that the data collected for one test is not affected by any other test, even if both the tests are set up on the same page and targeted to the same set of visitors.
Some tests may have variations that would be absurd if presented together. For example, one test is running to see the effects of discounts on specific products, while another compares performance when a blanket discount is provided. There is a chance that a visitor can experience variations where they get a discount for specific products and are also presented with the blanket discount from the other test. These visitors have a better upside and hence you have introduced a bias in both the tests. You can avoid that by making the test mutually exclusive, you can ensure that visitors are shown either product-specific discounts from test 1 or blanket discounts from test 2. This will help ensure that improvements in conversion rates from specific campaigns are attributed correctly to that test.
To know more about Mutually Exclusive tests in VWO, refer to this video.
Using the Mutually Exclusive Groups option in VWO, you can group multiple tests together that are mutually exclusive. By ensuring that your visitors are not exposed to multiple tests running on the same page, you can keep the reports clean, attribute the change in conversion rate to the correct test, remove bias, and avoid overlapping visitors across tests.
Many VWO users set up multiple tests on the same page and want them to run simultaneously. In such situations, a visitor on the page can become a part of multiple tests running on the same URL, and the visitor data is collected from all the test reports.
For example, let's say you want to run a test by changing the color and position of your primary CTA button on the homepage to make it more prominent on the page. Simultaneously, you also want to run another test on the same page by changing the headline of the page to test if an idea is communicated effectively to the users.
In such a scenario, if you run both tests at the same time to track the increase in engagement on the page, you cannot be sure which change delivered positive or negative results. In other words, you can not say accurately which set of changes had the maximum impact. Creating mutually exclusive groups allows you to run these tests without overlapping visitors' data between the tests.
For unrelated tests, it does not matter if your visitors become a part of more than one test. However, if the tests are related, you may want to keep the visitors exclusive to each test when you are:
- Running multiple tests on the same page
- Running multiple tests on the same funnel where there is a possibility of visitor overlap (like checkout or form pages)
Create a Mutually Exclusive Group from Campaign Settings
To create a mutually exclusive group at the account level:
- Click the gear icon ⚙ on the top right, and go to CAMPAIGN SETTINGS.
- Under the Mutually Exclusive Groups section, click Create mutually exclusive group.
- Enter the name for the group, and then select campaigns you want to include in the group. You can add a maximum of 10 campaigns in a group. You must have a minimum of two campaigns in a group.
- Click Create Campaign Group.
To add your test to a mutually exclusive group:
- In the last step of the test creation process, under Advanced Options, select Add to mutually exclusive group.
- Select an existing group where you want to add the test. You can create another group by clicking Create New Mutually Exclusive group.
- To add the test to the exclusive group, click Finish.