Mathematically speaking, even a very small improvement over the baseline can be considered to be an uplift, and given enough visitors, it will be declared statistically significant. However, from a business perspective, not all improvements are worth deploying, and we have found that experimenters often ignore such infinitesimal changes. The case is the same for any equivalent decline, as well.

Such a region where any minuscule improvement or decline is regarded as equivalent to the baseline’s conversion rate is called the Region of Practical Equivalence (ROPE). Businesses can configure the ROPE for their campaigns individually for the primary metric, depending on their scenario.

How to Declare Your ROPE?

Assume that your baseline conversion rate (aka the baseline average) is 40%. You have decided that keeping the 40% value as the median, any conversion rate between 38% and 42% will be regarded as equivalent to that of the baseline. For this, ROPE is calculated as follows:

Step 1: Obtain the difference between these individual extremes and the median. Here, the difference is ± 2.

Step 2: Divide the difference by the baseline average.

Now, ROPE = ± 2 / (40%) = ± 5

You can declare ± 5 as the ROPE for your campaign in this case.

What’s the Significance of ROPE?

There are three main ways in which VWO uses ROPE to deliver value to customers. Note that a higher value of ROPE will lead to an increased intensity of these benefits but also has some tradeoffs that are discussed in the next section.

• Declaring when to stop a variation: Using ROPE, VWO is able to give an early decision to disable a variation if it detects that the variation does not have the potential to be better than the baseline.

• Minimize False Positives: Without the concept of ROPE, we might mistake random fluctuations in data for significant improvements. ROPE acts as a safeguard against false positives, helping us focus on changes that truly matter.
  
  
• Saving Visitors on Average: The inclusion of ROPE delays the decision of a winner by some margin but makes the decision of disabling faster. Usually since winning campaigns are lesser in number as compared to those that do not end in a winner, ROPE saves you some visitors on average.

Considerations for ROPE

Determining the width of your Region of Practical Equivalence (ROPE) is a nuanced decision that depends on various factors, including the nature of your experiment, the desired level of confidence, and the practical significance of the changes you seek.

• Default Values: A conservative ROPE value of ±1% is a good default to start with, as it will save you a good amount of the False Positive Rate. VWO has configured this by default.
  
  
• Understand Your Business Context: Consider the unique aspects of your business and the target metric. Some businesses might have smaller margins of acceptable change, while others may tolerate larger values. Your understanding of what constitutes a meaningful change in your specific context will influence the value of your ROPE.
  
  
• Iterative Refinement: A/B testing is an iterative process. Initially, you might start with a broader ROPE and then refine it based on the results of your experiments. This adaptive approach ensures that your ROPE aligns more closely with the actual impact on your business.

In total, ROPE is a vital feature in the optimization toolkit for making informed decisions based on statistically significant and practically relevant data. After all, the essence lies not just in making changes but in implementing those that truly matter for a more optimized and seamless online experience.