In this article, you’ll learn about:
|
Sometimes, you may want to run multiple campaigns for the same cohorts. But you do not want to experiment with all of them with the same visitors. You may wish to keep the campaigns exclusive to one another. This is specifically helpful when you’re looking to experiment with two parameters for their impact on the conversion when exposed to the same population.
For example, one test is running to see the effects of discounts on specific products, while another compares performance when a blanket discount is provided. There is a chance that a visitor can experience variations where they get a discount for specific products and are also presented with a blanket discount from the other test. These visitors have a better upside, and hence you have introduced a bias in both tests.
You can avoid that by making the test mutually exclusive; you can ensure that visitors are shown either product-specific discounts from test 1 or blanket discounts from test 2. This will help ensure that improvements in conversion rates from specific campaigns are attributed correctly to that test.
Using the Mutually Exclusive Groups option in VWO, you can group multiple tests together and keep them mutually exclusive. By ensuring that your visitors are not exposed to multiple tests running on the same page/screen, you can keep the reports clean, attribute the change in conversion rate to the correct test, remove bias, and avoid overlapping visitors across tests.
Many times, you happen to set up multiple tests on the same page/screen and want them to run simultaneously. In such cases, a visitor on the page/screen can become a part of multiple tests running on the same page/screen, and the visitor data is collected from all the test reports.
For example, let's say you want to run a test by changing the colour and position of your primary CTA button on the homepage to make it more prominent on the page/screen. Simultaneously, you also want to run another test on the same page/screen by changing the page’s headline to test if an idea is communicated effectively to the users.
In such a scenario, if you run both tests simultaneously to track the increase in engagement on the page/screen, you would not be sure which change delivered positive or negative results. In other words, you can not say accurately which set of changes had the maximum impact. Creating mutually exclusive groups allows you to run these tests without overlapping visitors' data between the tests.
For unrelated tests, it does not matter if your visitors become a part of more than one test. However, if the tests are related, you may want to keep the visitors exclusive to each test when you are:
- Running multiple tests on the same page/screen
- Running multiple tests on the same funnel where there is a possibility of visitor overlap (like checkout or form pages)
Creating a Mutually Exclusive Group
You can create a mutually exclusive group of campaigns at the following two places in VWO:
From Account-level Settings
The Campaign tab on the Settings page has a section dedicated to creating mutually exclusive campaign groups. To create a mutually exclusive group from here, perform the following steps:
- Click the gear icon ⚙ on the top right, and go to Campaign.
- Under the Mutually Exclusive Groups section, click Create mutually exclusive group.
- In the popup that appears, enter a name for the group by clicking on the Edit group name icon.
- Under the Platform section, choose the platform for which you’re creating the mutually exclusive group. You can choose from the following:
- Website - Allows you to apply the group for website-based campaigns such as A/B test, Split test, Multivariate test, Personalize, and Rollouts.
- Mobile - Allows you to apply the group for mobile app A/B testing campaigns.
- Server - Allows you to apply the group for server-side testing campaigns such as Feature Rollouts, Feature Tests, and A/B.
- Website - Allows you to apply the group for website-based campaigns such as A/B test, Split test, Multivariate test, Personalize, and Rollouts.
- For Mobile and Server platforms, simply select the campaigns to be included as members of the mutually exclusive group using the dropdowns under the Select campaigns section. You need at least two campaigns to form a group. To add more, click Add Campaign. You can add a maximum of 30 campaigns in a group. Finally, click Create.
- The Select campaigns section for a Website platform will allow you to add the campaigns to the mutually exclusive group in the following ways:
- Random - This is the default option, through which the added campaigns will be equally and randomly distributed to the eligible visitors.
- Advanced - Through this, you can configure the distribution of the eligible visitors to the added campaigns by prioritizing or assigning weightage to them.
- Random - This is the default option, through which the added campaigns will be equally and randomly distributed to the eligible visitors.
- If you want to use the Random option, you can simply include all the campaigns that you intend to have as members of the group by selecting them from the dropdowns. You must have a minimum of two campaigns to form a group. To add more, click Add Campaign. You can add a maximum of 30 campaigns in a group. Finally, click Create.
- If you wish to tailor the distribution of eligible visitors to the campaigns, click on the Advanced option.
- Upon doing so, you will be able to see the following sections:
- Priority - You can use this option to prioritize your campaigns, by which the campaigns based on the order of priority will be served to the eligible visitors.
- Weightage - If you do not want to prioritize your campaigns or you have a bunch of campaigns that are prioritized and another bunch to be served randomly based on their weightage, you can use this option.
INFO: The visitors will be served the campaigns with the priority starting from the first in the list of campaigns under the Priority section, passing down to the ones in the Weightage section. The priority is denoted by P1, P2, P3,... By default, the Weightage section is assigned P1. It gets updated as campaigns are added under the Priority section.For example, consider that a new visitor arrives at your website and is found qualified for one or many campaigns in a mutually exclusive group. If the visitor qualifies for at least one of the campaigns in the Priority section, the first matching campaign will be served to them. If the visitor qualifies for only the Weightage section, they will be served a campaign randomly and based on their assigned weightage.
NOTE: When it comes to the server side, once a campaign from a mutually exclusive group is selected to be served to a visitor, based on their qualification, VWO compares the selected campaign with the called campaign. If they are the same, then the visitor is made part of the campaign, and a corresponding variation is served. Else, the visitor is not made a part of the campaign, and no variation is served.
- Priority - You can use this option to prioritize your campaigns, by which the campaigns based on the order of priority will be served to the eligible visitors.
- To add campaigns under the Priority section, click on Add Campaign. By default, the last added campaign will be given the last priority. But you can change the priorities by dragging the campaigns to move them up and down on the list.
- For the campaigns in the Weightage section, you will be able to assign weightage to the campaigns using the slider or directly entering the percentage of weightage. These campaigns will be served to visitors based on the weightage assigned to them, given that the visitors do not stand eligible for the campaigns in the Priority section. To reset the weightage distribution to split equally among the campaigns, click on Reset to split equally.
INFO: You can still create a group with only the campaigns in the Weightage section. - To delete a campaign from the list, use the Delete icon corresponding to that.
From Campaign-level Settings
Practically, you can add a campaign to a mutually exclusive campaign group at any time. You can either do this operation on a campaign that is getting created or on a campaign that is already existing. Indeed, you can even create a mutually exclusive campaign group right from here.
To add your campaign to a mutually exclusive group, perform the following:
- In the last step of the campaign creation process, go to Configuration > More configurations > Mutually Exclusive, and select Add to mutually exclusive group.
- From the dropdown, you can either select an existing mutually exclusive group to which you want to add the campaign or create a new group altogether by clicking on Create a mutually exclusive group.
- If you just want to add the campaign to an existing mutually exclusive campaign group, click Save after selecting the group from the dropdown.
- If you have clicked on Create a mutually exclusive group to create a new group, you will witness a popup wherein you have to configure the visitor allocation and the campaigns to be included in the mutually exclusive campaign group (as explained above from step 5).
- You can add more campaigns to the group by clicking on Add Campaign.
- To delete a campaign from the group, use the corresponding Delete icon that appears along the dropdown.
- Finally, click Create.