Rhett NortonOver half of your site elements don’t affect conversion rate the way you think they do. How to find out which elements help and which hurt your conversion rate? The solution is existence testing.
This isn’t a problem only with websites
I once worked with a company where everyone was convinced that one member of a team was the main value driver for the team’s multimillion dollar project. At the same time, this employee had a coworker that most of the team thought wasn’t bringing much to the table.
However, when the All-Star employee unexpectedly quit, the project suddenly went into overdrive. It turned out that—despite all the value that had been attributed to her—she had actually been holding the team back.
A few months later, everything ground to a halt when the “unimportant” employee left. To everyone’s surprise, his efforts had been critical to the project’s success and his knowledge and skills proved irreplaceable.
Everyone liked employee #1. Therefore, everyone believed she was producing value. No one thought much of employee #2. So, no one attributed much worth to his contributions. Were either of these assumptions founded in good data? No, but the whole team believed that they knew where all the value was coming from.
This happens all the time
This sort of story is far from uncommon. As human beings, we tend to assign value according to people or things that we like—how else do you explain our obsession with celebrities? We’ve never met these people, but we believe all sorts of positive or negative things about them based off how they are portrayed in the media.
Assigning value to the wrong team elements slowed and eventually crippled a multimillion dollar project. Are you doing the same thing to your website?
In digital marketing, things don’t always add up the way we expect them to. Sometimes, we think the equation should be 1+1+1=3, but the results come out as 1+1+1=2. In reality, the equation is never as straightforward as we’d like it to be and, as testers, it’s our job to figure out what’s really going on.
Which elements are helping your site? Which elements are hindering it? Discovering the parts of your site that aren’t contributing their fair share of value is critical to optimizing your site and realizing amazing testing gains.
Existence Testing
Removing a person from the work environment allows you to see what they were contributing. You see both the positive and the negative effects of their output when their contribution stops.
The same idea works for your website. To discover what the contribution of pages and elements to your bottom line is, we use a simple but strategic type of testing we refer to as “existence testing.” Simply put, we remove elements of your site and see what happens to your conversion rate.
Existence testing—it’s a good name for a very strategic way to learn about your site.
How Does It Work?
At its heart, existence testing is a relatively straightforward concept. You remove elements one-at-a-time to see how the removal of that single element changes visitor behavior. Only one element is removed for each variation so that everything is isolated.
By testing to see what happens if they don’t exist, you isolate the individual contribution of that element or page which makes it easy to attribute a change in behavior to that one element.
What Do the Results Mean?
If, compared to the control, the conversion rate improves when your banner is hidden, the banner was hurting your conversion rate. If the conversion rate decreases, the banner was helping you get conversions.
In essence, this is much like what happened with the company I mentioned previously. When element #1 (the favorite employee) was gone, things improved. The element was hurting the team. When element #2 (Mr. Nobody) was gone, the project fell apart. Mr. Nobody was the element that was helping the team.
Data Driven Decision Making
The great thing about existence testing is that it gives you the data you need to make the right decisions. Inevitably, you’ll discover some of your favorite elements are like employee #1—you might like them, but they’re hurting your business. On the other hand, there will be the elements that are unexpectedly, sometimes inexplicably driving major portions of your revenue.
What to Test with Existence Testing?
Okay, if you’ve made it this far, you’re probably sold on the idea of existence testing. The question is, how do you do it?
I’ve got an existential moment for you—you’re already using it.
If you think about it, existence testing is about more than just determining the value of page components. You can test any aspect of your conversion funnel, but two of the easiest areas to test are page existence and element existence.
Page Existence Testing
Most companies existence test their own homepage without even realizing it.
How? Landing pages.
A landing page is an existence test on the homepage. Often a good landing page that maintains the scent for potential customers has a higher conversion rate than the more generic homepage, so most companies assume that they’re better off with a landing page.
Well, duh, We All Know About Landing Pages…
Conventional wisdom, right? Years of experimentation support the idea that landing pages can be effective, but how many companies test the first page the visitor sees in an unbiased way? Do they test their landing pages against their homepage against other parts of their funnel pages? By challenging this assumption we’ve found that many companies are sending their traffic to the wrong starting location.
For some, the homepage is an important contributory part of their conversion funnel, but for others it isn’t as important. Without existence testing, these companies’ homepages would have been left in the same situation as the over-valued employee #1 from our story.
Even if you test this and you find your homepage is the best location you can still learn some valuable things. For example, a few years ago when I was testing with Redbox we wanted to find out what type of page was more likely to increase rental rental revenue. The home page looked like this:
Our test set up was this:
- Control: Homepage as the first page
- Variation 1: Movies Browse Page as the first page
- Variation 2: Games Browse Page as the first page
- Variation 3: How it Works Page as the first page
- Variation 4: Location Finder Page as the first page
- Variation 5: Top 20 Page as the first page
- Variation 6: Coming Soon Page as the first page
If you are curious to know which page won, all you have to do is go to redbox.com to see the live winning variation.
Every company and every site is different and you never know which landing page is the best one for your company until you test it out. This test gave us valuable learning that helped inform future home page tests.
We learned that the combination of movies and games was the best experience and that people needed to see movies they could rent before they saw locations or how it works or even movies coming out soon.
Page Level Existence Testing Summary
Unfortunately, you don’t know which starting location is best for your visitors until you test it.
Should you send traffic to your home page or to a custom landing page? Should you start visitors deeper in the funnel or higher up? Should your home page be part of the flow after a landing page? The only way to answer these questions is through testing.
So, if you want an easy place to start existence testing, look at the entire flow of your site. Which page should be the first step? How are different pages contributing? What happens if you remove a step? What if you add a step? Page-level existence testing can provide powerful insights into the effects of dramatic changes to your sales process. Additionally, it is usually a very efficient initial test to begin because you are simply redirecting traffic to existing pages.
Element Existence Testing
Once you’ve got a good sense for which pages contribute most to your conversion process, it’s time to get more granular. By hiding specific elements on a page and comparing conversion rates between the original and the modified page, you can determine how individual elements on a page are affecting your customers.
Let’s suppose you want to understand the importance of 3 elements on your homepage—your hero banner, form submit fields, and your search bar. To set this up you would have 4 experiences—a control and 3 variations where each variation hides just one element. It would look like this:
- Control – home page with no changes
- Variation 1 – hide just the hero banner
- Variation 2 – hide just the form submit fields
- Variation 3 – hide just the search bar
That Sounds Nice, But Does It Work?
After a series of existence tests on a client’s homepage, we discovered that eliminating their sliding promotion header increased revenue-per-visitor by 25%. When we hid the left navigation bar from one side of their page, revenue went up by 19%. Eliminating these pet design elements increased profits by $ 2 million a year!
After running this test we learned that the left navigation was hurting revenue so we decided to use that real estate better. The next test we ran changed the left navigation to be very different visually and as a result we saw a 17% increase in revenue-per-visitor.
The best part about existence testing is it gives you a great starting point for follow up tests because the previous testing output gave a prioritized ranking of what matters most.
A sister company to the company above ran a similar test and found something surprising. For the company above, the left nav was hurting the experience. For the company below the left nav was very important to the shopping experience. Hiding the left nav showed a decrease in revenue per visitor of -33%. This helped us learn the left navigation is very important to this company and that there is a lot of potential to optimize it.
We’re not the only ones to see impressive results from existence testing. EA removed the promo banner from their SimCity microsite and saw a 43% improvement in purchase rate. Impact eliminated their sidebar and increased their conversion rate by 71%.
What actions can you take after a test like this?
In addition to directly improving your conversion rates, page element existence testing can be a phenomenal idea generator. Once you know what works, you can replicate those elements on landing pages or elsewhere on your site. You can also try to milk site components by tweaking things like size, copy, color, imagery or location.
Overall, element existence testing is a great way to identify low-hanging fruit on your website. You’d be surprised how big of a difference seemingly small changes to your site can have on your overall profitability.
This can also be worthwhile for page elements that don’t currently have much of an effect on conversion rate. For those elements that are hurting your site, it’s often worth your while to remove them completely or test a replacement.
Adding New Elements
Existence testing helps you learn about the value of current site elements, but you can also use the same principle when adding new elements to discover which elements are driving value and which ones are the duds. Although it’s possible that you’ve achieved perfection with just a few tests, there are a whole host of new or improved elements yet to be tried that could drive significantly more conversions.
For example, SurvivalLife.com started their website with the kind of clean, minimalistic design most experts recommend. However, after testing their website, they eventually discovered that a messy website sold far better to their target audience. This revelation ultimately helped them to build their website to $ 1 million/month in sales.
Conventional wisdom isn’t always data-driven wisdom—existence testing shows you what is working, which is not always what common sense says should work.
As humans, we tend to believe that what works for us works for everyone. However, to run a successful website, you need to figure out what works for your target audience, not for you. That’s the power of existence testing.
Conclusion
When done right, existence testing is a phenomenal technique for improving your website. Whether you’re pulling apart your sales funnel or maximizing conversion rates from a page, existence testing shows you which elements of your conversion process are contributing or detracting from your success. Using data to eliminate the bad and maximize the good can do wonders for your company.
It’s just too bad we can’t existence test employees.
What about you? How do you feel about existence testing? Did this post spark any ideas?
Digital & Social Articles on Business 2 Community(175)
