You’re sure that changing your headline will draw more customers. Or you’re convinced that making your product description more prominent will encourage more purchases. Maybe you’re toying with the idea of making your offer bright red so that it will be more attention-catching.

But are you sure that these changes will improve your results? Relying on your own intuition can be enticing, but your assumptions may be faulty. How can you guarantee that you are basing your marketing on hard facts, rather than your gut?

There is a way to figure it out.

It’s a conceptually simple idea that every business should utilize, but few actually do.

The method is called split-testing. Using this technique can give you solid answers, backed up by data, about which of your ideas will be profitable — and which will actually be detrimental to your bottom line.

 

Split Testing Basics

Split testing is a method of controlled testing in which you test two different options, determining which will best increase a given metric (e.g., sales meetings, purchases, or website visits).

To create a split test, you will need to create another version of the marketing you’re testing, changing only a single variable. This will leave you with two versions, A and B. You will use Version A with part of your audience, and Version B with part of your audience. You’ll watch what happens, and carefully monitor which is more successful.

Just like a controlled science experiment, split testing can help you determine cause and effect, giving you solid proof to back up your conclusions.

You can use split testing to analyze the success of almost any change, but here are a few variables that are commonly tested:

• CTA. Your Call to Action prompts your prospect to do something specific. Where should the CTA be located on your website? How about the CTA in your emails? What color, font, and text size will convert best? Which wording will be most successful? Should your CTA be formatted as a button, or as hyperlinked text?

• Headlines. A visitor to your website spends just a few seconds before deciding whether to stay or go. An intriguing headline can make all the difference between a prospect continuing or bolting.

• Offers and Promotions. Would your customers be more likely to buy if you offer free shipping or a discount? Would they prefer a BOGO a half-price offer? Would a free initial subscription lead to  higher or lower retention when the free trial expires? With split testing, you can make sure that you’re making the best offer possible.

• Product Descriptions. Will detailed product description increase your conversion rates? Or will the details clutter the page, overwhelming customers and leading to a decrease in conversion rates?

• Email Subject Lines and Article Titles. Do your customers abhor clickbait? Or does an intriguing title draw them in? Your titles and subject lines are the primary clues that your customers use to determine whether to click and share your content. Why wouldn’t you test them out?

Even very minor changes, such as a product image or the time of day that you send out your emails, can drastically increase your performance. So no matter how perfect your marketing seems, split testing can go a long way towards helping you improve it.

For example, before the launch of its popular video game SimCity 5, EA (Electronic Arts) decided to design a promotion to encourage potential customers to pre-order the game. The offer was displayed as a prominent banner right at the top of their webpage. But the company wasn’t sure whether the promotion was actually increasing the number of pre-orders they received.

EA decided to conduct split testing to determine whether a different layout would increase their pre-order sales.

They took the original version and tested several variations with different layouts. They were shocked to find that the version that completely deleted the promotional offer drove the highest number of pre-orders! In this case, simpler was better.

Sure, it seems logical that a promotion would increase sales. But with split testing EA discovered that the opposite was true: when customers wanted to buy the game, they were most likely to do so from a page uncluttered with promotion text.

Split Testing in Nine Easy Steps

Now that you’re sold on the idea of split testing, it’s time to get started. Here’s a step-by-step guide on how to improve your marketing through split testing.

1. Pick a variable to test. Because you want your results to be conclusive, make sure to focus on only one variable at once. For example, you can first analyze the impact of your CTA’s background color. Wait for a subsequent split test to change the placement of the CTA. (Disclaimer: there are times where testing multiple variables is the goal, such as when you’re testing the combination of background color and text color. There is a more complex process called multivariate testing. But that’s beyond the scope of this article.)
2. Create a control and a challenger. After you have created your original (the control), you will need to create a variation (the challenger) in which the variable you are testing has been changed. The control and the challenger should be identical in all other ways.
3. Decide on the metric. Although you can measure multiple metrics during split testing, you need to determine your primary goal. For example, if your primary metric is conversion rate, you won’t be thrown off by the fact that your clickthrough rate was high; if your conversion rates fall, a high clickthrough rate is meaningless. Conversely, if your main goal is to increase website traffic or raise brand recognition, you won’t be thrown off by the fact that your clickthrough and conversion rates haven’t budged.
4. Decide on a sample size. On the one hand, your sample size needs to be large enough to make sure that you have statistically significant results. On the other hand, if you use your entire email list for the testing, you won’t have anyone to use your results on. Remember that your winning version will eventually be sent to the rest of your email list. The same thing applies to a webpage or social media post: you may continue the split test for a very short amount of time before deciding on a winning version. The time you keep your test running will determine your sample size.
5. Create your groups. Your A group and B group should be both equal and random. For example, your mobile device users and your desktop users should be divided evenly between your two groups. If they are not divided evenly, your results can be severely affected. For example, customers on a mobile device may be less likely to click on anything than those on a desktop; alternatively, mobile users may be drawn to clicking on one button size, whereas desktop users may be more likely to click on a different button size.
6. Choose a Split Testing Tool. Facebook,Twitter, and Instagram have built-in dashboards to analyze the performance of your ads. Many email marketing tools also allow A/B tests. To test your website or landing pages, you can use the Google Analytics Content Experiments tool. For the most robust testing, though, there are dedicated tools such as Optimizely and Unbounce that offer a huge array of split testing options.
7. Collect Qualitative Data. Split testing can provide you with plenty of quantitative information, but it’s important to under the reasons behind the data. Why did users prefer one version over another? You can find this out by adding an exit survey on your website, or a feedback request on your thank-you page.
8. React. If one version increased your goal metric significantly, then your split test has succeeded. Going forward, use the best option only. If you found no statistically significant difference, you can conclude that the variable you tested doesn’t impact your marketing.
9. Conduct another test. Your split test has concluded… but there’s more to come! You can usually find ways to further optimize your marketing. Design another split test, using a completely different variable. Bit by bit, you’re making your marketing better and better.

For example, when the insurance company Humana used split testing to determine which of two different ad versions should appear on their webpage, they discovered that the second, uncluttered version was better than the first, more detailed version. The second banner actually drove a 433% increase in clicks over the first.

But they didn’t stop there. Instead, they conducted additional split tests, which showed that adding a clear call to action button that read, “Get started now” instead of “Shop Medicare Plans” increased the ad’s clickability further, by 192%!

So why is split testing so important? It can convert your viewers into customers; allow you to focus your time, energy, and money on advertising efforts that really work; and generate more successful content for your business. Once you become accustomed to split testing, the notion of blindly creating content without testing its effectiveness will begin to seem more and more foolish. Instead, you’ll be using split testing effectively, easily achieving the highest quality results that are supported by empirical data and controlled tests.

---

 

This article is part of the series, So You Think You Know Marketing? These article shed new light on marketing fundamentals, with examples and tips to help you apply these concepts to your own business. See all the articles in the series here.