A/B Testing: Executing Tests to Get Results

There’s a reason why your website exists: you want your visitors to purchase the products they see. But, how do you build a site that encourages and incentivizes your users to do this? How do you determine what on your site is working and what isn’t?

The way to answer these questions? A/B testing.

The goal of A/B testing is to offer you insight on your customers and what they want from your online experience. Not all of your tests will yield insight, and not all of the winning site features you’ve implemented will help when combined, but A/B testing will help you determine what you need to do and what you need to change to optimize the experience you deliver to your customers. Here’s how to do this.

Step 1: Identify Problem Areas

The first thing you should do when beginning A/B testing is to gather information about your site as it is, so that you can focus your time and resources on areas that are negatively impacting your conversion rate.

Using some type of analytics tool (such as Google Analytics), you’ll want to identify specific problem areas on your site. For example, you might notice from your reports that your homepage has a high bounce rate or a certain area has a low click rate. Make note of these issues and begin your testing on these areas.

If you can’t identify specific areas that are problematic, you can breakdown your testing based on the actions your users have to take before they complete their transactions. These include:

  1. Navigating to your site
  2. Finding the product page they’re interested in purchasing
  3. Reading information about and looking at pictures of your product
  4. Adding your product to the shopping cart
  5. Provide the information needed to complete the transaction, such as payment and shipping information

Each of these steps presents an opportunity for A/B testing, keeping in mind that you’ll need to  tailor your testing areas based on variations in the traffic you see to maximize the effects of changes you make as a result of your A/B testing.

Step 2: Study User Behavior

Once you have identified problem areas on your site, study the behavior of your users on these areas. There are plenty of user behavior analysis tools available, such as Adobe Analytics, Appsee, CleverTap, or UsabilityTools, and these will help you pinpoint why specific areas of your site are problematic. For example, you might discover that the reason why you have a high bounce rate on your homepage is because your users can’t find the search bar to look for the items they’re interested in, or that a link that was once prominent is no longer visible due to a stylesheet change that you implemented.

Your goal is to put yourself into your users’ shoes, identifying any pain points or less optimal design features.

Step 3: Run Your Tests

Once you’ve gathered data on your site’s weaknesses and how your users are interacting with those areas, you’re ready to begin your tests.

The first step is to create an alternate version of your site to compare against the existing version. However, be sure to identify explicitly what you’re trying to test and making only that change, so that you can be sure that any differences in conversion rate can be attributed to that change.

Push this change to your site, and serve it to selected customers (the remaining customers will see the original website).  Over a certain period of time (say, one month), gather data on how each variant performs.

You should be certain to test each change you want to make to your site in isolation. Testing each step individually removes uncertainty that would remain if you made multiple changes to your checkout process simultaneously.

Step 4: Draw Conclusions

A/B Testing: Executing Tests to Get Results

At the end of your testing period, analyze your data to determine what happens next. If your tests show that there is a change that clearly results in a higher conversion rate, you can go ahead and implement that feature for all users.

If, however, the results are inconclusive, you can rerun your initial analyses and rethink the information you’ve gathered regarding your site’s problem areas and your users’ typical behaviors. Doing this prepares you for another testing cycle that hopefully yields actionable information.

Sitewide Testing

In the end, optimizing each change individually should lead to a more streamlined shopping process overall, but be sure to do one final test on the whole process once all the changes have been implemented. Compare this data with data gathered before you began A/B testing to ensure that the sum is not less than the parts.


Above all, your goal when doing A/B testing is to focus on micro-conversions (such as those resulting from small changes such as increasing the size of a call-to-action area) to achieve macro-goals (such as an increase in your bottom line). Though A/B testing can be time consuming, this insight is invaluable in gaining advantages over your competitors as the marketplace gets more and more crowded. Don’t miss out on making those sales due to your website!

To learn more about A/B testing, register for eTail West 2017 in Palm Springs, California! The three-day conference will provide detailed info on how to test your site for results, and will also explore other best practices for those in eCommerce. At eTail West 2017, you’ll see a ton of big names in the industry. Rigor will be at the conference in the South Foyer at table no.2. See you there!