Enable Theme Testing on VWO Shopify App

Overview

Shopify is an e-commerce platform that enables individuals and businesses to create and manage online stores. Website owners who run their online stores on Shopify often look for ways to improve user experience, boost conversions, and stay competitive.

One way to improve user experience is to experiment with new themes. Here’s where the VWO Shopify App comes into play. It creates a seamless connection between VWO and Shopify and automatically deploys VWO's SmartCode across your online store's web pages and new themes.

Use Case - Theme Testing with the VWO Shopify App

Let’s take a real-life use case for a better understanding:

A luxury ornament brand using Shopify observes a decline in the overall engagement and conversion rate. The website owner hypothesizes that changing the overall theme with a fresh UI and color schemes will enhance conversions. 

Here’s where the VWO Shopify App enables the website owner to conduct a Split URL Test to determine whether the new theme works better. The website owner simply needs to enable Theme Testing, and VWO automatically embeds its SmartCode in the theme under test. 

The website owner can then set up a Split URL Test to compare the performance metrics of the new theme against the current one. This approach to experimentation offers the following benefits: 

• Eliminates the need to embed the SmartCode explicitly on the new themes, saving time and effort.
• Ensures consistent data collection across new theme variations.
• Helps store owners test and optimize new themes seamlessly using VWO's Split URL testing feature.

Let us now understand how Shopify store owners can enable theme testing.

Enable Theme Testing on the VWO Shopify App

Install the Shopify VWO App and follow these steps to enable theme testing:

1. Log in to your Shopify account.
2. From your Shopify admin view, navigate to Online Store > Themes. 
3. Find the theme you want to test from the Theme library and click Customize.
   The system auto-generates a preview URL for the selected theme. It contains the theme ID you will need while setting up the split URL test in VWO, as shown in the following image.
   
4. From the top left, navigate to App embeds. 
5. Turn on the VWO SmartCode toggle. This activates the VWO integration with your Shopify website.
   If you have installed multiple apps, you can search for the VWO Shopify App directly in the search bar. This helps you access the VWO SmartCode toggle faster.
   
6. Open the toggle dropdown and enter your VWO Account ID. You can find it near the top right corner of your VWO dashboard after you log in.
7. To enable theme testing, turn on the Enable Theme Testing toggle.
8. Click Save.

Set Up a Split URL Test for Theme Testing in VWO

A split URL test lets you compare the performance of the theme variations hosted on each URL. After enabling theme testing in Shopify, you must set up a Split URL Test in your VWO account to test whether the new theme performs well.

To configure a Split URL test for your new theme:

1. Log in to your VWO account.
2. Go to Testing > Split URL. 
3. If you're setting up a Split Test for the first time, click Create a Split Test. If you've previously created Split URL Tests, simply click Create.
4. In the Control field, select the URL matches pattern option and enter the URL of your current Shopify theme as follows:
   Your_Shopify_Domain*
   
   The asterisk mark (*) lets you execute your campaign on multiple URLs. For example, you can use the pattern http://www.example.com* to run the campaign on the entire website. Similarly, you can use the pattern http://www.example.com/products* to run the campaign on your website's product pages.
   
   
5. In the Variation 1 field, select the URL matches pattern and enter the URL of your Shopify store with the theme ID of your new Shopify theme in the following format: 
   Your_Shopify_Domain*?preview_theme_id=XXXXXXXXXXXX
   You can find the theme ID after clicking Customize on the theme you want to test in the Theme Library. For more details, refer to step 3 in the Enable Theme Testing on the VWO Shopify App section.
6. Append a dummy query parameter (test=1) after the theme ID to the variation URL in the following format:
   Your_Shopify_Domain*?preview_theme_id=XXXXXXXXXXXX*&test=1*
7. Select the Exclude some URLs from the campaign checkbox, select URL contains, enter the query parameter (test =1) from the previous step, and click Save Now. 
   
   ATTENTION: Steps 6 and 7 above are required to be done to avoid the situation of an infinite loop for the visitors assigned to Variation 1. Here's why:

   • A split test starts, and a visitor is assigned to Variation 1.
     The visitor is redirected from the existing theme (Control) to the new theme (Variation 1).
   • Shopify automatically removes query parameters, such as 'preview_theme_id', from URLs when matching the currently loaded URL with the designated control URL within campaign settings. This removal makes the Variation URL identical to the Control URL, leading to an unintended redirection of the visitor back to the control URL.
     This creates an endless loop where users bounce between Control and Variation 1.

   • Adding test=1 to the variation URL and excluding it from the control URL pattern prevents the infinite loop by keeping it distinct from the Control, ensuring the test runs smoothly.
8. Click Next, set the Traffic split for your Control (existing theme) and Variation (new theme), and click Save Now.
9. Click Next and define the metrics you want to track to compare the performance between the two themes.
10. Segment your audience if you wish to target the audience based on specific criteria,
11. Before starting the test, use the Live Preview feature from the ellipsis menu to preview both the Control and Variant 1 URLs. This ensures that both the existing and new themes render correctly and allows you to validate the overall campaign.
12. As a final step, click the Start test button to run the test.

The image below showcases a sample Split URL Test comparing the Control (existing theme) with Variation 1 (new theme) for the product category pages.

Based on the Split URL Test results, you can determine which theme works best for your website and make data-driven decisions.