Smart Decision is recommended when there is no clear winner after a test is run. VWO suggests Smart Decision when the absolute potential loss associated with deploying the leading variation is less than the threshold of caring. The “threshold of caring” depends on the prior values entered while creating the primary goal.
The threshold of caring = Current conversion * Expected lift in conversion * multiplier * 100
where multiplier = 0.01 for High Certainty mode; 0.075 for Balanced mode; and 0.2 for Quick Learnings mode.
For example, if the current conversion rate is 10% and the expected lift is 20% with a balanced mode, the threshold of caring will be:
0.1*0.2*0.075 = 0.0015
When the potential loss of deploying a variation is small enough to be within the comfort threshold, SmartStats recommends that it’s a Smart Decision to deploy this variation.
VWO tests can have multiple smart decisions whenever the potential loss falls below the threshold of caring for your variations. You can choose to deploy the variation having the least potential loss value. By deploying Smart Decision, you will most likely not lose anything and in the best case scenario, you might even get something out of it.
For example, if the conversion rate for Control lies in the 10–20% range and the conversion rate for variation 1 lies in the range 20.19–27% range, variation 1 is declared Smart Decision because the difference in the upper limit of Control and the lower limit of the variation is less than 1%.