What Is Website Split Testing? An Introduction to Split Testing

It’s a fact of life that when people hire a web designer, they don’t just want a website, they want a website that does something! There can be a world of difference between these two things. The “action” they need the website to take for them can be one of several common things: selling products for their business (an e-commerce site), generating sales leads, and/or providing free information in the hope that the visitor will make a purchase from the company at a later date.

 Each company’s definition of a “lead” is different based on their web goals, so it’s important to businesses to be able to track leads! These aims differ slightly for some sites, like non-profits, blogs, and communities; however business sites, in general, aim to generate revenue by one of the above methods. Sometimes the client will split up these functions between two separate businesses – the website development will be taken care of by “Best Ever Web Designers Incorporated”, while the marketing is done by “Your Ad Here Pty Ltd”. However, if the push towards action (marketing) is created at the web design level, rather than externally, it is much more effective. The more effective your client’s website, the happier they will be; and the happier your clients, the better your web design business thrives! More than that, you can create an additional ongoing revenue stream that requires a little effort each month at your end but can provide a significant benefit to your client in the form of increased sales or sales leads for their business.

Here we look at how to determine which aspects of your web design work are most effective to help the client’s website achieve its aims by a process known as split testing.

What is split testing?

In website split testing (also known as A/B testing), a software application splits the traffic between two or more variants of the same web page, measuring whether there is a difference in the effectiveness of that page in achieving the website’s goals. Every time a visitor takes the desired action on a website, such as making a purchase or submitting a form, it is called a conversion. At its core, split testing, along with the data it provides, is used to determine the version of a page that converts best.

Web designers might check whether:

• A landing page with a graphic of a person encourages better conversion rates than one with a graphic of the product
• Different positioning of the shopping cart button encourages more people to click it
• A form with fewer elements, or different elements, encourages more people to finish filling it out.

Just about every aspect of a website can be split-tested. Split testing involves a single element of a page being changed at one time. A different type of split testing is sometimes used to achieve faster improvements with a shorter number of tests; multivariate testing.

In this type of testing, multiple variables are changed within a single web page. So, rather than having multiple versions of the same page, multivariate testing rotates different elements on the same page, for example, the headline, the call-to-action, the image, and the opening paragraph.  Both multivariate and split testing have their advantages.

The advantage of multivariate testing is that you can find the optimum combination of website elements in a shorter space of time and achieve significant improvements in the website’s performance; the disadvantage is that it takes a considerable amount of traffic to get accurate results. For most websites, A/B split testing will be the optimum method of improving a website’s conversion rate over time.

Why should I run split testing on my website?

Split testing is what separates elite web designers from the rest because it provides them with the knowledge of which combination of website elements will produce the optimum results for the client and they can demand a higher rate as a result. Split testing can help:

• Get visitors to stay longer on the website
• Get visitors to interact more with the site
• Get more visitors to request a consultation or join a newsletter
• Get more visitors to buy from the site

It can also achieve these aims (depending on what the site is set up for) at a lower cost per action (CPA) than competing sites in the same niche. In some ultra-competitive online markets (home loans, insurance, accommodation, SEO, online dating, etc) split testing is the only way to guarantee that you will be heard above the noise of the competition. Split testing also helps ensure that when it comes to advertising, you are maximizing each and every dollar.

If your website doubles its conversion rate from 1% to 2% using split testing techniques, you have just doubled your revenue using the same marketing budget. You will be able to spend more on advertising your site and earn a better return on investment (ROI) from each dollar you spend. Sounds like you’d be crazy not to, doesn’t it?

It becomes even more exciting when you start to examine the numbers. Many untested websites have conversion rates that run around 0.25%. This means that it takes 400 visitors to get one sale.

With split testing, you don’t need to increase the number of visitors, just the rate at which they convert. Many success stories tell of conversion rates upwards of 10%. That would be the equivalent of a 40-fold increase in visitors, all of a sudden giving you 40 sales or sales leads out of 400 visitors, instead of 1.

Who should be using website split testing?

If you have a site you are trying to increase traffic to, whether it be a hotel, restaurant, or any other type of business, you should be split testing.

Ideally, the web designer should start split-testing as soon as the site launches, as they are the ones who have designed the site and have all of the source files on hand. Even if the owner feels that the website is performing well enough, the benchmark for “good” performance could be moved even higher by using split testing. Just keep in mind that split testing does require an established traffic flow because you need people visiting your site to be able to split-test web pages.

Why is it ideal that the designer does it at the launch stage? Because it is much easier to keep the site as a work in progress after launch, not only visually and structurally, but in terms of its goals. It is infinitely harder for a third party to come along later and change things around, and in many cases, you wouldn’t want them to.

That site’s your creation! Doing the split testing yourself, as a designer, also gives you a real-world feel for what elements will work and what ones won’t. It adds an extra dimension to your sense of aesthetics, removing the guesswork that you necessarily engage in during the design phase.

You are no longer doing what you think will create conversions, but implementing what you know produces measurable results. And this knowledge alone will increase the value you bring to your future projects.

Setting up a split test with Google Website Optimizer

Google Website Optimizer is an A/B split testing and multivariate testing platform that allows you to easily see the effects that making changes to page elements has on your website conversion rate. It’s a free platform and is robust and user-friendly.

One of the major advantages, though, is that you can use the Google Website Optimizer with existing web page URLs (e.g. yourdomain.com); some other split testing applications require you to use a custom URL (e.g. yourdomain.com/tracking.php) which makes it very difficult to split test non-paid website traffic, such as SEO traffic, referral traffic or direct traffic.

Google Website Optimizer serves different pages to different users, with hundreds of variations possible. In our experience, using the website optimizer tool does not affect your rankings and does not cause duplicate content issues. Here is the process for setting up an A/B test with Google Website Optimizer: Identify the page or pages you want to test (usually the home page or pages that receive a lot of traffic).

Define which element you want to test. In A/B testing, you’ll only test element one at a time. Commonly tested elements are the headline, sub-headline, opening paragraph, image, call to action, and the submit button (look and/or placement).

Add tags to the original page, the variation page, and the conversion pages to allow tracking. Enter the website URLs you are testing into the Google Website Optimizer. Preview how the pages will look before sending them live.

Google Website Optimizer will rotate the page that is displayed for every visitor so that the first person sees version 1, the second person sees version 2, the third person sees version 1, and so on. You can access the results and reports of various statistics within Google Website Optimizer to see which variant is the most effective.

What should you test with google optimizer?

These are the most commonly tested elements of a web page:

• Headings
• Sub-headings
• Opening paragraph
• Images
• Call to action
• Offers
• Hyperlinks
• Submit button
• The location of different elements
• Colors

The first six elements are the ones that generally have the most impact on conversion rates, and the ones you should test first. Google once tested 41 different shades of blue for their pages. They have the luxury of doing this because they get more traffic than any other website, and they have the drive to do this because performance is their life.

They thrive on it! You don’t need to go to this extent but if Google places so much value on it, shouldn’t you?

Knowing when a split test is finished

The number of conversions needed to garner a reliable indication of future traffic varies incredibly with different types of sites. If the business you are designing for is a high-customer volume, low-value spend store, you should test until you have 70 or so conversions for one variant.

If it is a low-customer volume, high-value spend store, aim for 20 conversions. If the set conversion is further from becoming an actual “profit” for the store (i.e. a conversion creates a lead from a visitor, rather than a sale from a lead), you may want to be more certain that one variant is working better than the other.

You may also want to test more different variants of the same element or do multivariate testing … within the limitations of your traffic volume.

Determining statistical significance at the conclusion of a test

For you to have a high confidence level that the test results aren’t just a fluke, you can apply a simple formula to determine whether the difference in results is statistically significant. This rule is: The numerical difference between the two results must be greater than the square root of the sum of the two results.

Or

Y-X>?X+Y, where Y>X

Sounds like a nightmare from before your high school maths exam, doesn’t it? It is easier to understand with an example:

• If the result of page A was 20 conversions, and the result of page B was 30 conversions, then the difference between the two is 10.
• The sum of the two results is 50
• The square root of that sum is 7 (rounded down)
• The difference (10) is greater than the square root of the sum (7), so the result is statistically significant.

If the two results had been 22 and 28:

• The difference between the two is 6
• The sum of the two results is 50
• The square root of that sum is 7 (rounded down)
• The difference (6) is less than the square root of the sum (7) – the result is not statistically significant.

If the results of your split test are not statistically significant, you can either:

• Keep testing and see if a more definite pattern emerges
• Decide that the element doesn’t make a difference to conversion in this case, and test something else.

Tips for ongoing split testing

Either you, as the designer, or the website owner, should be implementing split testing constantly, on all your high-traffic web pages. It takes time to get test results, and in the meanwhile customer preferences and market sensibilities are changing, along with the season and your stock. It is important to note that not all split tests will be successful.

In fact, if you have a 20% success rate where one in five split tests improve the overall conversion rate of the site then you are doing well. The key point to remember with split-testing is that every success forms a new baseline, with considerable website performance improvements achieved after a number of different split tests. The easiest way to manage all of the split tests is to set one day per month when you review results and then set up tests for the coming month based on those results.

Every month you should track your results in a spreadsheet to avoid accidentally re-testing the same elements over and over again. This also provides a point of reference for when you are starting new projects – you can see which placements and copy types are most effective in which industries. If your website grows to the point where you have upwards of 1 million visitors per month, you can use multivariate testing.

And don’t forget to check what competing sites in the same niche are doing, and test the same elements on your site! Once you start split testing you’ll find that what you thought would perform better doesn’t and you’ll be very surprised with some of the results. The best part about split-testing is that the confidence you will gain from split testing will help you with your future web development projects because you will have increased knowledge of what produces results and what doesn’t.

Consumer behavior is a specialized field and web designers can get on a steep but exhilarating learning curve for it by engaging in split testing on their own and their client’s websites.

Related Content

• 15 Tools for Monitoring a Website’s Popularity
• 7 Incredibly Useful Tools for Evaluating a Web Design
• 6 Tools to Help You Analyze a Web Host
• Related categories: User Interface and Web Design