Home Research A/B Testing – An Overview

A/B Testing – An Overview

Published on October 9, 2012 by in Research

By Marc Niola | October 8, 2012

A/B Testing

A/B Testing (Source: UX Acrobat)

A/B testing or split testing compares the effectiveness of two versions of a web page, marketing email, or the like—in order to discover which has better response rate, better sales conversion rate, or the like (Source: Wikipedia). Several researchers and marketers use A/B testing to gain valuable insights about customer’s behavior and improve ROI.  A/B Testing has been known to dramatically decrease bounce-rates while increasing conversion-rate by significant margins. A/B testing is gaining momentum in the UX community as a formidable testing practice that builds success.

Getting started with A/B Testing

A/B Testing is a website optimization technique that consists of sending a percentage of users to one version of a page, and the another percentage to a different page. After analyzing the traffic with web analytics tools, researchers should be able to discern which page is performing better than the other one. Below is a short video from an Australian SEO workshop on the benefits of  A/B testing.

Once your website testing strategy has been evaluated, and A/B Testing is deemed the most appropriate method to meet your goals, you need to align your A/B Testing strategy with your objectives. For example, if you want to increase user feedback, concentrate on the elements that directly affect users submitting a form, email or tweet. By narrowing down the elements that control ‘user feedback’ in this example you can try several different methods for users to send feedback and observe which one gives the best results.

How it works

Most A/B Testing tools are back-end facing and split incoming web traffic to different versions of a particular webpage. After the testing period has elapsed, conversion rates are analyzed to see if statistically significant differences in conversion are present.

Benefits Of A/B Testing

A/B Testing is versatile because it can be used to test several different elements of a website’s conversion path. If you want to evaluate different button styles for a call to action, restate a value proproition, try different headlines or images, A/B Testing can be quickly changed to test almost any current vs alternative conversion path element. It can also be used to test trustworthiness and expertise, two components that together form credibility and trust the hallmark of any successful business. This flexibility can create a testing frenzy, and it is important to stay well focused to the best results.

A/B Testing Diagram

A/B Testing Diagram (Source: UX Acrobat)

Using A/B Testing for multi alternatives is not the best strategy. For instances like this there are other tools to choose from. Remember to keep your testing goal in mind when designing and evaluating A/B Testing results. Your conversion path design choices should be made methodologically and backed-up with research that supports your final design decision.

Top 3 Take-Aways

  1. Trust Test Results: Researchers are often surprised that less-than aesthetically pleasing elements produce better conversion rates. Remember most of the time you will be testing to how to achieve a desired goal. If the ugly bright orange button produces better conversion rates than stick with it. Don’t become anchored and emotionally invested in any one design. We all have our favorites, but A/B Testing is about data-driven results not what looks good.
  2. Always Test Current and Alternative at the same time: You have to test your alternative against a control (current version) to make testing valid. Not following this basic tenet will skew your results and could hurt our bottom-line.  Always test current and alternative versions simultaneously .
  3. Stay focused and consistent: Develop a test plan and target your objectives.




Written by  – The UX Acrobat

Follow Marc on Twitter: @MarcNiola




About UX Acrobat - Marc Niola

Marc is an expert UX professional with a focus on customer-centered design principles, social media, gamification and psychology. He has developed innovative business solutions for global corporations, SME, agencies, clients and brands.

Connect with Marc on

 Share on Facebook Share on Twitter Share on Reddit Share on LinkedIn
No Comments  comments 

Leave a Reply

Your email address will not be published. Required fields are marked *