How to Perform an A / B Test for Product Recommendations - Step by Step Instructions
Google Optimize is used, inter alia, for conducting A / B tests. Such tests are used to verify which version of the page is bringing the best results (results can be measured in terms of, for example, the number of orders placed, the average length of a user session, etc.). Such tests are definitely worth carrying out in online stores to decide which sales support solutions give the best results.
In today's post, we would like to present you instructions on how to conduct an A / B test on product recommendations in an online store.
Sample A/B test scenario
We want to choose the banner on the home page so as to encourage the user to stay on the website of our store for as long as possible. We have three ideas concerning the look of this banner, but we don't know which one will work best and which will attract attention of users the most. We can check it with the A / B test.
We create three versions of the home page, each with a different banner and run the test. During its duration, the user who visits the home page is randomly displayed one of the versions. Then, his behavior is recorded on the website (in particular, the duration of the browsing session, which ends when the user leaves the store).
We end the test when we collect the right amount of data – say, when the home page is visited by 6,000 users during the test and each version of the page will be displayed 2 thousand times. After completing the test, we can check for which version of the banner the length of the user's session is the longest and insert this banner on the page as the final version.
Our instruction covers how to run Google Optimize and how to set up an A / B test that will check which version of a product page - with or without our recommendations - guarantees more "add to cart" actions.
A / B test for product recommendations - step by step
IMPORTANT: all the steps below must be performed in the Google Chrome browser, being logged in to the email account connected to the Google Analytics store (Google Analytics is necessary).
Install the Optimize Chrome Extension plug-in in your Chrome browser, available at this link.
Create an account in Google Optimize by visiting the website. Fill in the fields appearing on the screens as desired, their content mainly concerns accepting the regulations and determines the extent to which you want to share data with Google on the website. This has no effect on the performance of the A / B test.
After creating the account, go to "Settings" in the upper right corner
a. In the Settings screen, link your Google Analytics account to the newly created Google Optimize account.
Select your store from the dropdown options and then click "Connect"
b. On the Settings screen, copy the script snippet there and paste it into the section of your store (similarly to the way you pasted the Recostream code).
Now you are ready to create an A / B test. Click the "Let’s go" button on the main page:
Fill in the form:
a) Choose the name of the experience (any name, e.g. Recostream test).
b) In the next field, enter the url of the editor's page, i.e. a sample page where you will define the tested changes on the page. In our case, this should be the url of any product displaying Recostream recommendations.
c) Select A / B Test as the experience type.
- On the next screen, find the "Objectives" section and click "Add Experiment Objective". Then select "Select from list" and decide which variant you want to measure, for example Add To Cart or Checkout complete.
- Now you can proceed to define the version of the page you want to compare. To do this, scroll to the top of the screen, click "Add variant", select the name and confirm:
- Go to the edition of the created version by clicking "Edit"
- You should be redirected to the website of the product whose url was indicated in point 5 b) as the editor's page, and an additional window should appear in the lower right corner (Chrome plugin installed in point 1). It looks something like this:
- Scroll down to where the Recostream recommendations appear and click the title bar:
- When the bar you clicked on in the previous point is selected, in the gray bar at the top click on the element shown in the image:
- In the window in the lower right corner of the screen, click "Edit Item" and select "Delete" from the list. The Recostream recommendation snippet should disappear.
- Save the changes with the button in the upper right corner, and then click Done (in the same place) to return to the administration panel.
- Now check the appearance of the original version and the changed version of the page. In order to do this, click "Preview" next to both versions. Make sure you see Recostream recommendations in the original version and not in the revised version.
- The last step is very important. Scroll down to 'URL targeting' and select the edit icon.
On this screen, define which pages you want to perform A / B testing on.
a) If you decide to do it on all available product pages (in this case you will get results the fastest), then you need to define a rule by which Google will know that a given url is a product page url. If each product page in your store has the following form, for example: www.my-store.com/product/product-name-id.html, this url can be recognized by the repeated fragment "/ product /". Then the rule might look like this: b) If you do not want to run tests on all product pages, you can either transform the rule so that it covers fewer products (for example, give the name of one of the categories as a text for comparison, if it appears in the url) or list specific product addresses that you want to conduct tests on. It is important to remember that the selected products generate some meaningful traffic (otherwise the test will take a very long time). For example, a rule where a specific product list is provided looks like this: After selecting the option that suits you, save the changes.
- We've reached the end. All you have to do is run the test. To do this, press the "Get started" button at the top of the page:
Distributing user traffic to several versions of your online store's test pages is an integral part of the website improvement process. When deciding on new solutions, such as, for example, a product recommendation engine, it is worth using Google Optimize and, based on the above instructions, independently check which version of the test is better received by the majority of website users. Conducting A / B testing is a great way to increase traffic and sales in your store.
In order to measure the effectiveness of recommendations on an ongoing basis and their impact on store metrics, we recommend our previous post, in which we provide instructions on how to connect the recommendation system with Google Analytics. You will find it under this button: