When doing their website development, few people use A/B testing. Some people are simply unaware of it while many don’t do it because it requires continuous effort involving multiple tests. Rather than do the work, they’ll focus on other things such as bringing in more traffic. This is unfortunate because A/B testing by itself will improve the profitability of your site. Once your website has sufficient traffic levels to make A/B testing possible, you should start your testing right away. Once your site has been optimized for conversions, then you will get a better return on investment from further traffic driving efforts.
A/B testing is conducted on a single element on a single page or a single element that is common across your entire website. Two versions of this element are tested against each other. One is your current version called the “control” and the other is your experimental version. Whichever version produces the best conversion result is kept. Before conducting more A/B testing on this element, you should do tests on other page elements to determine which elements result in the biggest conversion changes. This will save you wasted effort on trying to optimize elements that have little to no impact on conversions. Once this is done, then further A/B testing can be done on the remaining elements.
The type of conversion that is measured in these tests can be anything you want. It could be the number of sales, number of sign ups to your newsletter, or the number of responses to a survey. Some elements commonly tested include photos, calls to action, short versions vs long versions of a page, video vs no video, pricing, headlines, and the layout of your site.
Some basic rules that will help with your A/B testing are:
Test Both Versions at the Same Time
Traffic behavior can be different depending on the day of the week and even time of day. To eliminate the effect of time, both versions are tested simultaneously. Therefore which version a visitor will see is randomly selected. In addition, the same visitor should see the same version on any repeat visits that he makes. This will require the use of specialized testing software.
Conduct Tests with a Sufficient Amount of Traffic
An A/B test based on two visitors is not a good test. So how many visitors are sufficient? That can be determined by a number of online applications such as this one. Basing your tests on the number of visitors recommended by the app will allow you to form statistically valid conclusions.
Trust the Results of Your Tests
Your common sense about which version should convert best is based on the preferences of one person: yourself. A sample size of one is not large enough for a test to be valid. After testing with a sufficient number of visitors, trust the test.