When we talk about hypothesis testing in the CRO world, we typically mean testing different ideas against an observed problem or opportunity. But do you think there could only be one or two ideas to test? Of course no. The list of ideas could be infinite, and to test each one of them, time and resources are required.
With this being said, one might have the following questions- Which hypotheses should be tested? Which ones should be deprioritized or just thrown away? Thus, you must figure out the potential hypothesis to test first and spend more time on, i.e., you need to identify the order in which they will be tested. In other words, you must prioritize your hypothesis.
In this article, we will explore how you can prioritize your hypotheses in VWO.
In VWO, once you have created an observation and formed a hypothesis around it, the next step is to prioritize each hypothesis.
Prioritizing helps you scientifically sort your hypotheses. The closer a hypothesis is to the top of the list, the higher is its priority. To prioritize your hypothesis, VWO comes with a rating system that helps you rate (higher or lower) a hypothesis you feel has the maximum probability of accomplishing the change in conversion towards the desired direction.
While you create a hypothesis, the option to rate it appears in the Prioritization Score section.
You can rate a hypothesis based on the following parameters:
- Confidence: On a scale of 1 to 5 (1 being the lowest, and 5 being the highest), select how confident you are about achieving the expected improvement through the hypothesis?
- Importance: On a scale of 1 to 5 (1 being the lowest, and 5 being the highest), select how crucial the visitor landing on the test pages (for which the hypothesis is created) is.
- Ease: On a scale of 1 to 5 (1 being the most difficult, and 5 being the easiest), select the hypothesis's complexity. Rate how difficult it’ll be to implement the changes identified for the hypothesis.
Things to Consider While You Rate Your Hypothesis
The purpose of prioritizing is to ensure that you are running the most relevant tests for the business and getting the best ROI on your tool and people. However, there are other factors too that you must consider when prioritizing tests:
- How confident are you of achieving the uplift?- Prototyping the user persona you are targeting can help you in determining the potential of a hypothesis. With a sound understanding of your audience, you can make an educated assumption on whether the hypothesis will address the users’ apprehensions and doubts and nudge them to convert.
- How valuable is the traffic you are running this test for?- Your website may be attracting visitors in large numbers, but not all visitors become buyers. Not all convert. For example, a hypothesis built around the checkout page may hold a higher importance than the one built around the product features page.
- How easy is it to implement this test?- Next comes determining the ease of implementing your test. Try to answer some questions: Would it require a lot of strategizing on your part to implement the hypothesis? What is the effort needed in designing and developing the solution proposed by the hypothesis? Can the changes suggested in the hypothesis be implemented using just the Visual Editor, or does it warrant adding custom code?