Now reading:
How to Perform an A / B Test for Product Recommendations - Step by Step Instructions

How to Perform an A / B Test for Product Recommendations - Step by Step Instructions

Avatar Michał Kosztołowicz
17 marca 2021
clock icon 12 min
Jak przeprowadzić test AB na rekomendacjach przy użyciu Google Optimize

Google Optimize is used, inter alia, for conducting A / B tests. Such tests are used to verify which version of the page is bringing the best results (results can be measured in terms of, for example, the number of orders placed, the average length of a user session, etc.). Such tests are definitely worth carrying out in online stores to decide which sales support solutions give the best results.

In today's post, we would like to present you instructions on how to conduct an A / B test on product recommendations in an online store.

Sample A/B test scenario

We want to choose the banner on the home page so as to encourage the user to stay on the website of our store for as long as possible. We have three ideas concerning the look of this banner, but we don't know which one will work best and which will attract attention of users the most. We can check it with the A / B test.

We create three versions of the home page, each with a different banner and run the test. During its duration, the user who visits the home page is randomly displayed one of the versions. Then, his behavior is recorded on the website (in particular, the duration of the browsing session, which ends when the user leaves the store).

We end the test when we collect the right amount of data – say, when the home page is visited by 6,000 users during the test and each version of the page will be displayed 2 thousand times. After completing the test, we can check for which version of the banner the length of the user's session is the longest and insert this banner on the page as the final version.

Our instruction covers how to run Google Optimize and how to set up an A / B test that will check which version of a product page - with or without our recommendations - guarantees more "add to cart" actions.

A / B test for product recommendations - step by step

IMPORTANT: all the steps below must be performed in the Google Chrome browser, being logged in to the email account connected to the Google Analytics store (Google Analytics is necessary).

  1. Install the Optimize Chrome Extension plug-in in your Chrome browser, available at this link.

    test A/B Google Optimize
  2. Create an account in Google Optimize by visiting the website. Fill in the fields appearing on the screens as desired, their content mainly concerns accepting the regulations and determines the extent to which you want to share data with Google on the website. This has no effect on the performance of the A / B test.

    test A/B Google Optimize for ecommerce
  3. After creating the account, go to "Settings" in the upper right corner

    Connect Google Optimize with Google Analytics
    a. In the Settings screen, link your Google Analytics account to the newly created Google Optimize account.

    Select your store from the dropdown options and then click "Connect"

    b. On the Settings screen, copy the script snippet there and paste it into the section of your store (similarly to the way you pasted the Recostream code).

    Google Optimize
  4. Now you are ready to create an A / B test. Click the "Let’s go" button on the main page:

    Why should you do an AB test?


  5. Fill in the form:

    a) Choose the name of the experience (any name, e.g. Recostream test).

    b) In the next field, enter the url of the editor's page, i.e. a sample page where you will define the tested changes on the page. In our case, this should be the url of any product displaying Recostream recommendations.

    c) Select A / B Test as the experience type.

    Why should you do an AB test?
  6. On the next screen, find the "Objectives" section and click "Add Experiment Objective". Then select "Select from list" and decide which variant you want to measure, for example Add To Cart or Checkout complete.

    Why run test A/B Google Optimize

    Why should you do an AB test?
  7. Now you can proceed to define the version of the page you want to compare. To do this, scroll to the top of the screen, click "Add variant", select the name and confirm:

    Settings of Google Optimize
  8. Go to the edition of the created version by clicking "Edit"

    How does the Google Optimize work
  9. You should be redirected to the website of the product whose url was indicated in point 5 b) as the editor's page, and an additional window should appear in the lower right corner (Chrome plugin installed in point 1). It looks something like this:

    AB testing
  10. Scroll down to where the Recostream recommendations appear and click the title bar:

    A/B testing for product recommendations
  11. When the bar you clicked on in the previous point is selected, in the gray bar at the top click on the element shown in the image:

    ustawienie google optimize dla testów AB
  12. In the window in the lower right corner of the screen, click "Edit Item" and select "Delete" from the list. The Recostream recommendation snippet should disappear.

    ustawienie google optimize dla testów AB
  13. Save the changes with the button in the upper right corner, and then click Done (in the same place) to return to the administration panel.
  14. Now check the appearance of the original version and the changed version of the page. In order to do this, click "Preview" next to both versions. Make sure you see Recostream recommendations in the original version and not in the revised version.

    Jak ustawić google optimize dla zrobienia testów AB w sklepie internetowym
  15. The last step is very important. Scroll down to 'URL targeting' and select the edit icon.

    Jak ustawić google optimize dla zrobienia testów AB w sklepie internetowym
  16. On this screen, define which pages you want to perform A / B testing on.

    a) If you decide to do it on all available product pages (in this case you will get results the fastest), then you need to define a rule by which Google will know that a given url is a product page url. If each product page in your store has the following form, for example: www.my-store.com/product/product-name-id.html, this url can be recognized by the repeated fragment "/ product /". Then the rule might look like this:

    How to Perform an A / B Test
    b) If you do not want to run tests on all product pages, you can either transform the rule so that it covers fewer products (for example, give the name of one of the categories as a text for comparison, if it appears in the url) or list specific product addresses that you want to conduct tests on. It is important to remember that the selected products generate some meaningful traffic (otherwise the test will take a very long time). For example, a rule where a specific product list is provided looks like this:

    Tests AB with Google Optimize
    After selecting the option that suits you, save the changes.
  17. We've reached the end. All you have to do is run the test. To do this, press the "Get started" button at the top of the page:

    How to Perform an A / B Test for Product Recommendations

Summary

Distributing user traffic to several versions of your online store's test pages is an integral part of the website improvement process. When deciding on new solutions, such as, for example, a product recommendation engine, it is worth using Google Optimize and, based on the above instructions, independently check which version of the test is better received by the majority of website users. Conducting A / B testing is a great way to increase traffic and sales in your store.

In order to measure the effectiveness of recommendations on an ongoing basis and their impact on store metrics, we recommend our previous post, in which we provide instructions on how to connect the recommendation system with Google Analytics. You will find it under this button:

arrow-up icon