Why You Suck at A/B Testing

Arielle Hurst

At this point A/B testing is a marketing tactic we’ve all heard of. When business owners look to optimize their website these kind of experiments are usually the first place they turn. So why beat a dead horse? Frankly, most businesses do A/B testing wrong. To be fair, it’s not really their fault. We’ve all been told to A/B test one thing on our website at a time so we can see how every alteration affects conversions. That makes sense, right? Sure, but it’s still a poor approach.

Let’s take a look at the stereotypical A/B test: a button color experiment. We want to know if the green or blue button converts better. We create two versions of the button, wait for a few weeks and at the end we have an inconclusive result that tells us blue increases conversions by .43 percent. Are you excited by that progress? Yeah, me neither. The reason this type of test doesn’t create real results is because most businesses haven’t refined their website messaging enough to test these granular details and get more than marginal gains.

If you shouldn’t just be testing one web page component at a time, what should you be testing? To find out, let’s look at a website doing the kind of A/B test we’re all used to.

Version A

Version B

A simple headline test. What’s the problem here? Well, to your website visitors these two versions are the same. Yes, they have different words on them, but they evoke the same emotion. Instead, experiment with something different.

New Messaging

This test is going to create a clear winner and loser because it appeals to your target customer in two very different ways. To figure out what connects with your audience most, keep experimenting with different messages that utilize a variety of emotional and logical appeals.

Once you’ve found the theme that’s most effective, exaggerate the concept to see if you can improve conversions even more. To get a feel for this, let’s look at an A/B test the talented team over at Copy Hackers did for their client Dressipi.

Original Messaging

Above is the message that most effectively connected with Dressipi’s ideal customer. With that knowledge in mind, the CopyHackers team dove deeper and researched what women were saying about their bodies and finding clothes online. By sifting through blog posts, forums and returned clothing reviews, they were able to “steal” new copy straight out of a potential customer’s mouth. Armed with the words women actually use to describe the shopping experience, they formed an exaggerated headline and call to action that authentically reflects the way Dressipi’s customers talk about clothes.

Exaggerated Messaging

Although the frankness may seem daring, this version of the website converted 14.3% better than the previous version. Amazing! If you want to create more authentic website copy, try looking at reviews of a competitor’s (or your own) product to see how people describe the positive and negatives. Honing in on key words and phrases will help you craft the type of headline that feels authentic and pulls in prospective customers.

At this point most of us would leave the website alone. Conversions have already increased considerably after all. Instead, the CopyHackers team created one more version that combined the original headline with the more daring call to action.

Conversion Champion!

Just the difference in button copy generated 130% more clicks to sign up than the original!

The takeaway?

Refining your message is the most important A/B test you can do on your website. Forget about button color until you have the essentials nailed down and don’t be afraid to go bold!

Have you ever conducted a really successful A/B test? What were results?

(137)