A hypothesis is an assumption ("I think X happened because...") that explains an observation (an issue) for further investigation. It is not necessarily true, but it’s a starting point of your optimization journey. Using a hypothesis lets the user examine the assumption and assess the likelihood of its success. Not only does this lead to improved performance, but it also saves resources.
In the world of website experience optimization, a hypothesis is an assumption that a proposed change in your website would increase customer engagement. The hypothesis includes details about why the observed issue happened, who does it affect, what parts of the website are impacted, and how well the issue can be solved by your hypothesis. The stronger the hypothesis, the higher are the chances of an uplift.
Since the hypothesis is formed before the test, it guides the design of your test, i.e., it helps you focus on what the customer wants (company goals), and based on that, you create and deploy a variation.
A strong hypothesis helps in:
- Thinking deep about the possible outcomes of an experiment
- Generating testable predictions
- Drawing relevant conclusions about a problem
The structure of a basic hypothesis follows a CHANGE: EFFECT (if...then) framework, i.e., “changing” one thing “into” another thing “will” result in something. The changes could range from something as small as a change in the headline or call to action button to more extensive things like a change in page layout or website flow.
If [we run this test] then [we will expect to see this] because [of this]
Example#1: Removing the optional fields from the signup form will increase
Example#2: Changing CTA from “BOOK YOUR FLIGHT TICKET NOW” to “TICKETS ARE
SELLING FAST - LAST FEW LEFT!” will improve the sale of flight tickets.
To ensure that your hypotheses can be proved or disproved by an experiment,
you must keep the following things in mind while forming a hypothesis:
- What change you are testing
- What impact you expect the change to have
- Who you expect it to impact (the target population)
- By how much (percentage change in the conversion)
The hypothesis lifecycle starts with a proposition that might fix the problems on your website. Once you have formed a hypothesis by listing down all the what and why’s, you make relevant changes to your webpage and create a variation of it. Next, you compare the performance of your variation with the original webpage.
Based on the test results, you can determine whether the answer/solution corresponds to what you had proposed earlier. If it does, your hypothesis is accepted, and you can deploy the variation for 100% of your visitors else you should reject the hypothesis and start again with a new hypothesis altogether; refine your hypothesis and do more tests.