The Importance of A/B Testing

What is A/B testing?

Also referred to as “split testing” or “bucket testing” – A/B testing is the method a marketer can use to compare two versions of a webpage, email or app, to see which one performs better.

By showing two variants of the same page to users at random, companies are able to see through statistical analysis, which variation performs better.

How does A/B testing work?

If you want to test a particular landing page on your website for example, then you’ll need to modify it and create a second version of that page. Here are the six steps for successful A/B testing:

1. Collecting the data
   Before you begin, you’ll firstly need to decide which pages you are going to test. Google Analytics is a great place to start. Identifying pages with high traffic is ideal, as they’ll help you to generate results quicker. Similarly, discovering pages that have really low conversion rates is useful, as it’ll give you an indication as to which pages could do with being improved.
2. Determine your goals
   To get the most out of your website A/B testing, you’ll need to set your goals. These could be anything from button clicks to form fill-outs, or email sign-ups. The goals will essentially form the variations you’ll be creating. For example, if you want to increase button clicks, one of your variations may be to change the colour of the button, and another may be to move it further up the page.
3. Generate your hypothesis
   The next stage, whether it’s for your website, your app or email, is to create a hypothesis for each and every single test you’d like to conduct. For example, you may want to change the colour of your CTA button from white to red, because you think it will increase the number of click-throughs. Once you’ve created the entire list, prioritise them in the order you want to test them. Those that have the highest expected impact and are the easiest to implement should be tested first.
4. Create your variations
   Once you’ve got your priority list, it’s time to start creating your variations. The original landing page (version A), known as the “control” will stay the same, whereas version B, the “variation” will have the change made. It’s very important that you stick to one test at a time, so your results are reliable.
5. Run the experiment
   Now your control and variation pages are set up, you can begin the experiment by using the website A/B testing tools we’ll explain below. 50% of your visitors will be randomly directed to your control page, whilst the other 50% will be directed to your variation page. Statistical analysis will now begin, determining which page performs better for your desired goal.
6. Analyse the results
   When your control and variation pages have gained enough traffic, the results will be ready to analyse, and you’ll be able to see if your hypothesis was correct. Depending on the results, you can either keep the original page the same, or make the change you tested. Once that’s been done, you can start the cycle with the second variation on your list.

Why use A/B testing?

The benefit of split testing is that it can help you to increase your conversion rate, by enabling you to make small changes over time, to enhance your user experience.

The statistical data you’ve gathered for each test is based on real-life users, and will help to either prove or dispel your initial hypothesis. Either way, website A/B testing will help you to create a more user-friendly site, whilst you gain insight into aspects such as how the colour of a CTA button impacts user behaviour.

Rather than running these tests as a one-off, it has many more benefits if you use it continuously; which is why it’s so important you create several hypotheses which you can test one after the other. By making one small change each time, you’ll have granular insight into the exact impact every change has made.

The importance of ecommerce A/B testing

If you run an ecommerce website and notice that your cart is getting abandoned often, and you’ve reported a drop in sales, then you may want to try out split testing.

Ecommerce A/B testing isn’t just limited to minor changes like the colour and positioning of a button. Instead, treat it as a way to remove as many pain points as possible, to make the transaction process simple for the customer. You could run tests on product photography, navigation organisation, product page layouts, promotions and more.

Amazon is a great example of successful ecommerce A/B testing. Through these tests, they removed the need for customers to repeatedly fill out their billing information, reiterated the quality of their products by placing the importance of reviews, photos and questions; and also made the filter functionality easier to use.

Recommended tools for website A/B testing

There are several tools that you can use to conduct these tests, so it’s worth trying them out to determine which one you find the easiest to use.

Optimizely

Optimizely is a great option if you’re just starting out with website A/B testing. Installation is simple, and plan prices are reasonable.

When you initially set up your platform, you’ll need to decide which experiment you want to run. Multivariate tests are an option (testing multiple changes on one page at the same time), as are multi-page tests (testing changes to multiple pages); but we’d recommend sticking to just A/B testing, so you can accurately track performance for every single element.

With Optimizely, you can change the elements on your variation page without having to spend too much time looking at your website’s code. You can also dictate the percentage of traffic that goes to your control and variation page. Whilst 50% is the standard, if you run an ecommerce site and there’s a potential your variation could lose you a considerable amount of sales, then you’ll probably want to direct less users to that page to begin with, so you can closely monitor it.

VWO

VWO is used for website testing by a range of high-profile brands including Virgin Holidays and eBay.

Like Optimizely, you can also run multivariate tests (which we wouldn’t recommend). The benefit of VWO is their reporting dashboard, which shows your results in a digestible, visual way.

You can also use their SmartStats feature, which allows you to run faster tests which you’ll have more control over, enabling you to reach more accurate conclusions.

Crazy Egg

In addition to conducting A/B testing, Crazy Egg’s offering extends to heat mapping (a visual analysis of how users are interacting with your pages), and usability testing tools.

If you don’t have any coding experience, Crazy Egg is a great option as you don’t need to touch the code to run effective tests.

Final thoughts

If you haven’t yet dipped your toes into the world of testing your site, then now’s the time. The key thing to remember is to only change one element to begin with, so you can ensure results are accurate.

Whilst you may have wondered how to do A/B testing, there are several tools that make it easy for you to set up and run tests, without you having to mess around with code.

By running split tests over the long term, you’ll improve the user experience on your website, which will help to increase your conversions – which is great news for your business.

For the latest news in digital marketing, head over to our Articles page. Alternatively, if you have any questions or if you’d like to know how we can help you with your digital marketing, get in touch today.