The Best A/B Testing Tools For Mobile Apps

A/B testing tools have their origin in web design, and the concept behind them is simple.

For example, you’ve got a website with a front page designed to get visitors to buy something. Sales aren’t where you’d hoped they were, and so you look at potential changes that might drive conversions higher.

Let’s say you’ve determined the “Buy” button was too small. You’ll then make a variation of the webpage that has a bigger button. When visitors get to your site, they’ll be randomly redirected to one of the two versions: Version A, which is the original site, or Version B, which contains the newly designed button.

After a time, you analyse the results to see what worked better. If the Variation improved sales, then that becomes your new home page.

That’s a simplified explanation, not the least of which is that A/B testing isn’t just about web pages. It’s the perfect tool for testing mobile apps, too—especially when the app’s purpose is to work as a mobile storefront.

But there’s an added challenge to testing on mobile—actually pushing changes out to your users.

On the web, it’s easy: make the change on the server, and anyone visiting after that gets served the new content. But how does that work on an app, which is downloaded by users and lives on their devices? Huge companies with the resources to do it can build out hardware and software infrastructure that can push changes in the app’s content instantly, but what’s a small business to do? Also, managing variation testing can quickly spiral out of control when you’re talking about multiple pages within an app, or multiple versions of the app depending on the hardware.

[Side note: Those of you playing the “There’s An App For That” drinking game at home should get ready to slug back the adult beverage of your choice, because it’s going to happen.]

A/B testing is only effective if you’re doing it right. So how do you do it? Well, even in the world of mobile app testing, there’s an app for that. Several, actually. Have a look at these five different app tools to help you get the most out of getting the most out of your mobile channel.

1. Apptimize

At the heart of any A/B testing tool is the ability to make instant changes to an app’s UI; If you had to wait for people to update the app to deploy variations, testing would be kind of a nightmare.

This is great for when you want immediate aesthetic changes to the user interface, maybe to correct a mistake or to add some seasonal flair to the look (think Holidays). Apptimize calls this “Live Updates” and it’s the engine that pushes test variations out to certain users.

For more complicated tasks, the app lets you get under the covers and start tweaking or removing entire code blocks—but it’s the Live Updates feature that makes this tool standout from the competition.

That’s because, useful as it is for A/B testing, Apptimize lets you use it as a mechanism for changes to your app’s interface, pushed out to users instantly. If you want to create a temporary change to the UI—like adding a holiday theme, or bringing attention to a 24 hour sale—Live Update has you covered.

And Apptimize doesn’t just manage your tests—it creates them and handles the rollout.

The user-friendly interface makes it easy to track all your changes during the creation of a test.

Normally, there’s a lot of behind-the-scenes coding that has to happen when deploying a variation—developers need to know which parts of their code have been changed for testing, which means going through that code and hand tagging each section that’s been changed.

With Apptimize, the hours needed to is removed from the equation. Simply make the changes you want to see, and Apptimize takes care of everything else. That kind of attention to saving time and headaches for coders is a huge reason for it being on this list.

Apptimize offers two different levels of subscription to its service, which consists of more than just A/B testing.

Small business owners will be happy to learn that there’s no limitation to the testing component on the smaller plan. Everything you need to design, deploy, and analyse your variations are included in the lower level plan, along with live updates, and feature flagging (good for monitoring rollouts of new versions).

2. Azetone

These testing tools all rely on the ability to instantly and seamlessly push updates to users. A/B testing is just one possibility, and its feature set usually follows a prescribed formula.

Azetone’s is as fully featured as you find, and it even integrates with Push Notification platforms but what differentiates their service is how they take advantage of the real-time ability to update apps.

Using Azetone, you can A/B test your way to highly personalised user experiences. Find out what works in a given city, then create content that’s dynamically served when a user’s in that city. Personalisation can be done down to the user, based on their profile, and Azetone integrates with third party CRM platforms to provide even deeper insights.

Azetone’s biggest selling point may be its analytics engine, though.

Of course, it provides data about conversion rates, and can even segment those down even further on a number of criteria—by user demographics, device type, location, all the things you’d expect.

What’s most impressive about Azetone’s analytics is how helpful they can be before you’ve even run your first A/B test. Just tell it the pages in your app you want it to monitor, and let the software do its work for awhile. Soon, you’ll receive detailed reports with heat maps showing the areas of each page where the most pressing/scrolling/zooming/etc. is happening. Along with these visual representations of where people are tapping around is a detailed report that boils it all down to actionable numbers.

This kind of insight into how people use your app is invaluable, and can even point you in the direction of the kinds of variations you might want to make for testing.

For a lot of businesses, they don’t have the money to invest in focus group studies, or the time to develop and analyse feedback questionnaires. Azetone pre-emptively strikes at the UI issues you didn’t even know you had, and then makes it possible to test the variations that might fix those issues. This aspect of Azetone is what makes it a real standout among its competitors.

3. Leanplum

Positioning itself as a platform to drive mobile engagement, Leanplum certainly has the features to back that up.

Its A/B testing module should more accurately be called ABCDEFGHIJKLM(you get the idea) testing. The tool’s support for multiple combinations of variants (multivariate testing) would add only a layer of confusion were it not for the comprehensive reporting that makes sense of it all.

Leanplum simplifies complicated multi-iteration testing in ways that other tools don’t even try to, and presents all the data to users in an instantly understable way.

At the core of Leanplum’s mission to drive engagement is its integrated messaging functionality.

With it, multiple channels—including in-app messaging, popups, push notifications, and emails—can be managed from within Leanplum and tightly woven into its testing and reporting.

It collects a lot of data about how effective each channel (and each message in that channel) is in engaging and retaining customers. But it goes further than that, by automating messaging on a personal level, and responding to user behaviors.

If someone keeps checking out a green dress but never buys it, for example, a friendly push notification or email can be sent out if the price drops. This automation can scale out across users, resulting in automated email drip campaigns and in-app messages designed to bring users back in. All of which can be tested, of course.

That attitude of “You’ve got the data, might as well act on it” is what makes Leanplum such an attractive choice.

4. Splitforce

The first on the list to take the “Do one thing and do it well” philosophy, Splitforce doesn’t offer anything apart from A/B testing tools—but they do it really well.

As with the others on this list, Splitforce needs no coding to make variations. Non-technical designers can easily tweak the UI or even turn entire features on and off. Meanwhile, coders can play around with custom elements and test entirely new code for things like new features or new levels to a game.

Behind all the testing is Splitforce’s data analytics, the idea behind which actually gets into some fairly heady concepts of probability and statistics that are too boring to explain here. Suffice it to say, Splitforce employs a machine learning algorithm that continually optimises its analysis with an ever growing number of data points.

The difference in conversion rates between the control group A and the variation B might only be a few percentage points off—essentially a statistical coin toss to human eyes. Splitforce’s internal deep dive into the performance of each variation allows to uncover the weakest variations earlier, and start automatically pruning them away and moving users over to the ones that perform better.

The good news there is that by the end of the experiment, there will be some pretty compelling and inarguable data, one way or the other, as to what variations work once the testing is complete.

Even better though, is that by eliminating weak variants from the pool and moving users to the better performing ones, the testing software is actually driving conversions and retention while the experiment is still running. No one has to do a thing to make that happen, and no other testing tool boasts this kind of intelligence.

Splitforce distinguishes itself in this regard, actually improving your app before you even had a chance to analyse each variations performance.

5. Swrve

The concept behind Swrve is that not all users are created equally. It’s not just a matter of taste and preferences, either—where someone is within the customer lifecycle should also determine the content they receive.

This is a feature that makes Swrve unique: a suite of messaging and engagement tools targeted to users based on these lifecycle criteria. In other words: a sweater loving 30-year old woman who’s just downloaded your clothing app shouldn’t be receiving the same messaging as her demographic twin who’s already a regular customer. At different stages in the lifecycle, they’re going to require different kinds of outreach.

From onboarding to monetisation, Swrve’s advanced testing tools will show you how to refine your app to increase signups, engagement, and sales.

Swrve maintains a database of all kinds of revealing KPIs: a built-in receipt verification system ensures that revenue data is a true reflection of your app or a campaign’s success; user experience data (like the number of sessions per user or average time per session) demonstrate exactly how and when people are using your app; customisable funnel reports show where in the app’s workflow you’re actually losing people, points which would benefit from good A/B testing.

All this data is used to continually personalise the app experience for each user, so that they’re receiving messages they’re likely to respond to, and offers they’re likely to take you up on. By offering insights and data that allow you to more personally engage with customers, Swrve stands as one of the top testing tools available.

Conclusion

You can see from this list that there’s more to A/B testing than just throwing some stuff out into the world and seeing what works.

In and of itself it’s an effective way to keep refining your app to make it the kind of thing people will want to have on their devices. Your app is only as good as the most recent experience each user has had.

Apps that are stagnant, or with updates stuck in review before being allowed into an app store run the risk of losing users. By changing, testing, personalising, and analysing quicker than ever before, you can ensure you retain the users you worked so hard to get in the first place. Fast iteration cycles are shown to be key to an app’s long-term success, because they never get the chance to become static. They’re too busy being improved.

While this kind of development strategy used to belong to huge companies with the resources to support it, that’s no longer the case. The app tools covered here are giving small businesses a chance to test their way to longevity, too.