Running Before You Walk: What You Need to Know About Personalization

May 6, 2016

Web personalization is all the rage, but are you trying to run before you’ve learned how to walk?

Don’t trip and stumble in your drive to master shiny, new techniques – instead, learn what you need to get accomplished first, and what the pitfalls are if you don’t.

The State of Personalization Today

Personalization is a buzzword that you hear mentioned in all of those articles predicting next year’s marketing trends. It’s on the tip of the tongue of every savvy marketing professional in cocktail party conversations.

Personalization – otherwise known as targeting, one-to-one marketing, predictive targeting, etc – is defined as “delivering different experiences to customers based on attributes of that customer.”

Adage found that, when asked to prioritize one capability that will be most important to marketing in the future, one-third of marketers answered: “personalization.”

Another study found that 94% of customer insights and marketing professionals across multiple industries said personalization is “important,” “very important,” or “extremely important” for meeting their current marketing objectives. And many studies suggest that consumers want this type of personalization online:

Image Source

However, Gartner found that less than 10% of tier 1 retailers believe they are highly effective at personalization, and nearly one-third report having limited or no capability to support personalization efforts.

In addition, we found in our 2016 state of the industry survey that 64% of our respondents weren’t using any kind of web personalization yet.

People think personalization is important, but they aren’t executing. Why the disconnect?

What’s The Rub With Personalization?

I’m not sure why more people haven’t tried personalization, but the reason many aren’t seeing crazy success is because they forgot about one big trade-off: cost.

With personalization, this cost comes mainly in the form of increased organizational complexity.

As Matt Gershoff, CEO of Conductrics, put it, “while targeting can be incredibly valuable, many in the industry haven’t fully grasped that targeting ALWAYS leads to greater organizational complexity, and that greater complexity means greater costs.”

According to Gershoff, there are three pieces to the personalization puzzle:

• Data
  
• Content
  
• Targeting Logic

1. Data

Data is, of course, the information you collect about your visitors. It gets more complex when you introduce the idea of personalization, though, in that you need to ensure that:

• You have accurate user data
  
• That it’s available to your marketing systems at decision time.

So you need to source the data, and make sure it’s of high quality, and also make sure you can pull relevant data at the right time in order to deliver personalized experiences.

2. Content

The content is also fairly straightforward – it’s the experience you deliver to visitors. If you’re not running any tests, you’re simply assigning one experience to all users. If you’re running an A/B test, you’re basically delivering two different experiences to two sets of users (divided at random for science).

But when you’re doing personalization, the amount of content you can assign is infinite, as the amount of users assigned to experiences is infinite as well. As Andrew Anderson, Head of Optimization at Malwarebytes, put it:

Andrew Anderson:

“I can change copy, I can change work flow, I can change layout or features of the experience. Even better, I can do this for the same user in a thousand different ways. I am a returning user to your site… but I am also a user in the afternoon, who came from Google, who has been on the site 12 times, who has made 3 purchases, and who is using FireFox.”

3. Targeting Logic

Finally, with personalization you’re introducing a new asset: targeting logic. This is how you connect the data with the experiences. You can use business logic (assuming certain segments should have certain values – you can even test them), or you can introduce machine learning to sort of set the learning on autopilot. But this part introduces a certain level of complexity that many overlook when thinking about personalization.

Where There’s Targeting There’s Complexity

“Complexity is the flipside to targeting”

– Matt Gershoff, CEO of Conductrics

Everyone has heard the benefits of a 1:1 experience, but no one thinks about the increased complexity with each new targeting rule, each new experience crafted.

Think about it – The more experiences you have, the more you have to manage. Your marketing system becomes more complex with each step. Matt Gershoff explained it really well in a Digital Analytics Power Hour interview…

Matt Gershoff:

“You’re always going to be making your system more complex.

It’s going to be more complex because now you’re going to have to manage all of these different experiences.

It’s also going to be more complex because you need to take into account the inputs, so you need to have data about your users at decision time and that data needs to be managed – because that data needs to be accurate and timely.

So you need to have a whole system managing your data…and you then have greater management costs for your content or your experiences. And it doesn’t need to be just presentation layer stuff, it could be some backend business rule or different pricing, it could be different sort algorithms, or whatever – they all need to be managed.”

By complexity, I don’t mean the technical aspect of creating an experience and serving it to a segment. There are many tools out there, it’s not a technically difficult thing to do.

The fact is, you can personalize anything, and that’s one of the most inspiring and difficult parts about personalization. As Andrew Anderson said, “the simple act of creating an experience is not the goal, the goal is to do so in the way that generates the greatest ROI for my organization.”

There seem to be two general reasons for the complexity inherent with personalization…

• The Assignment Problem
  
• The Management Problem

1. The Assignment Problem – How Do You Make The Rules?

Targeting is an assignment problem. As Gershoff put it, personalization is fundamentally the “process of assigning users to experiences.” It’s the taxi-cab problem in machine learning. How do you predict, optimize, exploit the optimal paths/experiences? How do you decide to assign X experience to Y segment?

Business Rules

One method people use is business logic/business rules (opinions).

That’s when, in simplest terms, you decide which segments have the highest impact opportunities and you test variations that you think will bring ROI. You can use segmentation in post-test analysis to find out if certain variations had higher impact areas on certain segments, or you can use less efficient means (opinions and guesses). That’s what many organization are doing right now – arbitrarily assigning.

This method, of course, has many downsides (most of which will be brought up in the optimization and management sections), but one of the biggest is the increased resource cost of manual assignment. As Matt Gershoff said in the podcast, “At one end of the spectrum, you only have one experience. At the other end of the spectrum, you have thousands of experiences.”

So to hope to find enough correlative data to find exploitable segments requires lots of work and even more error.

Optimization/Discovery Rules

To that effect, Andrew Anderson recommends a process of discovery and exploitation that is a sort-of continuation of his discipline-based approach but applied to segments based on the discovery phase (as follows)…

• Create multiple executions of the message
  
• Take 2 to 3 other messages that could be used there (one will most likely be your default content), other concepts such as specific products or specific site offerings. Hopefully you are just reusing existing content.
  
• Serve all the offers to everyone
  
• Look at the results by segment and calculate the total gain by giving a differentiated experience:
  
  
  
  • If you are correct, then the highest performing recipe for the previous shopper segment will be one or both of the shipping messages. Default content would then be the winner for the non-purchaser segment (the comparable segment).
    
  • If you are wrong, then any other segment will have a higher winner for any of the offers. Be open to permutation winning that you never thought of. Being wrong is always going to provide the greatest return

Then there’s the exploitation phase, when you actually get the value from your discovery – push live the highest revenue producing opportunity found.

As Andrew says, this is fundamentally different than those who fail at personalization, which looks like this usually:

• Push the single piece of creative to the repeat purchaser segment.
  
• Hope

So that’s really the difference between discovery based targeting logic and opinion-based. Discovery based arises from an already mature approach to optimization and simply calculates gains from potential different experiences, then exploits in those segments as well.

Machine Learning Rules

Reid Bryant from Brooks Bell agrees that basic segmentation falls short because it depends on an analyst making an arbitrary decision on which dimensions are most relevant to targeting. How to assign experiences then? The other optimization is machine learning rules and algorithms. Here’s how Bryant put it…

Reid Bryant:

“A better approach is more algorithmic: an analyst uses unsupervised learning techniques, such as clustering or latent class analysis, to determine the number and definitions of each group. The algorithm will only partially answer the question—the business will need to determine if it’s profitable to actually direct distinct campaigns to each group.”

The Management Problem: Maintenance and Optimization in the Long-Term

Another of these trade-offs is the maintenance cost. Here’s how Andrew Anderson put it:

Andrew Anderson:

“There are maintenance costs for each experience that you maintain – meaning that if you have 3 dynamic rules on a page, you are now maintaining 3 user experiences, not just one. This makes future testing harder and can lead to a very mismatched site if you don’t find similar exploitable segments on other parts of the user experience.”

A good way to think about personalization is as a step from and a contrast to A/B testing. Matt Gershoff said it really well in the Data Analytics Power Hour podcast…

Matt Gershoff:

“The reason people get excited about A/B testing is it’s like the first step into having your marketing application have multiple applications.

Even if it’s not conditioned on the user, your site can now be in more than one state, in a way. You can have an A experience and a B experience. It’s almost like you’re in the multiverse. You’re living in one world and an experiment lets you live in multiple worlds. And then you can see which world is doing the best, and then you can collapse all the other worlds and just live in the best world.

With targeting it’s different – you keep open these other universes. And they’re always there because some users are going to do better in those experiences. So you have a greater complexity to manage.

I think that’s the reason why, even though people have been going on for more than 20 years about 1:1 marketing, many of the people listening and many companies don’t employ it: there’s a complexity, a cost.”

Management costs are not limited to on-page optimization; you’ve got to think about it holistically. Matt Gershoff gave a great example of customer support complexity…

Matt Gershoff:

“Let us say you have a customer who contacts us with a question about a digital experience they had. In the pre-targeting world, it was fairly easily to determine what the customer’s experience was. With a targeted marketing system, what they experienced is now a function of both their ‘data’ and the targeting logic that were both active AT THE TIME of the experience.

So whereas before, introspection was trivial, it is now extraordinarily difficult, if not impossible in certain contexts to discover what experience state the customer was in. And this difficulty only increases as the system complexity is a function of the cardinality (number of options) of both the customer data and experiences – the finer the targeting the greater the complexity.”

What About 1:1 Personalization?

1:1 marketing is supposed to be the holy grail of personalization; delivering a unique experience for each individual.

Image Source

But you can already see that with the additional complexity of each new experience, it’s a hypothetical possibility but a logistical improbability. At least that’s how Reid Bryan from Brooks Bell sees it, saying the marginal utility of each subset decreases due to the “costs of unique products, the expense of unique creative design, and the digital burden of modified user experiences.”

Matt Gershoff, too, said that “targeting is usually a marginal thing. It’s not a winner take all thing.”

You get a baseline by doing one experience for everyone and start to get big lifts by running two different experiences for highly exploitable segments. But again, as you chase smaller and less exploitable segments, it becomes a game of inches.

It’s not to say personalization can’t be worth it; just that you need to think about costs, benefits, and marginal value…

Image Courtesy of Matt Gershoff and Conductrics

How Do You Know Your Personalization is Working?

Another management problem is tracking the ROI of your personalization? How do you know if it’s working?

If you’re just arbitrarily creating targeting rules, you don’t know anything outside your current exploitation. Without the discovery phase, it’s tough to tell if personalized experiences are moving the needle and by how much.

Andrew has a great story about two times where, by using his strategy above, companies were saved from potentially negative effects of personalization…

Andrew Anderson:

Two of my favorite stories for this involved one group trying to target based on a very advanced statistical analysis they did, with three identified points in the user experience.

We just took 3 versions of the content they were going to serve, 2 other pieces they already had around, and tested them out. If they had targeted the way they wanted to, they would have lost 18% of leads across their entire site, but the same test showed that by serving based on time of day gained them 20%, and that was just from one dynamic change with one of the other pieces of content we threw in.

The other story was from a travel/hotel group that wanted to serve localized content to people based on geolocation. We instead served to everyone and their east coast properties won for everyone, providing a 14% gain to bookings across their entire site.”

All Ideas Are Perishable

Finally, there’s the perishability of solutions (which you also have to deal with). Perishability means that, yes, for a while your solution may be optimal but could fade in effectiveness over time (due to many things).

When asked whether he thought referral engines (such as those on eCommerce sites that recommend related products) suffer from regression to the mean, Matt Gershoff responded that it’s more a matter of external factors and perishability….

Matt Gershoff:

“I think the world has drift. We’re in a world in which things change over time. Our solutions are perishable in many situations. I think that’s another key thing to think about – the perishability of the problem. It’s something you should ask before you start trying to solve a problem. If it isn’t very perishable, that’s a situation where an A/B test approach is going to be effective.

It could be that, for movies, they’ve seen lots of films, new ones are coming out – let’s say you’re Netflix and Amazon has it’s own products – there are lots of things outside of your control in the real world. Your users could start to look different over time. Just in general, the world changes from underneath you.”

So you can never really stop optimizing (or managing your personalization program). The more experiences you’ve created, the more you’ve gotta worry about maintaining, tweaking, and improving in the future (understanding the complexity now?).

Why Do Most Companies Fail at Personalization?

According to Andrew Anderson, many companies do targeting just for the sake of targeting – which is meaningless and is often negative to their bottom line.

He actually avoids the term ‘personalization,’ referring to it instead as ‘dynamic user experiences.’ According to him, it helps remove some of the inherent baggage with the buzzword. Here’s how he explained common problems with companies jumping into personalization too fast and without a plan:

Andrew Anderson:

“Without the culture of discovery and of really wanting to go where your users want to go many companies end up just targeting based on some random dimension that they have available and then call it a day. Not only does that take away resources from future tests, it makes it harder to test in the future and it makes harder for that group to really understand just how far targeting and personalization really are.

One of the things that it takes the longest time to understand is that the things that you need to target on are never the things you think you need to target on. I have worked with over 100 different groups trying to do “personalization” and I have not once had the initial targeting idea prove to even be valuable let alone the best option. Mature test programs are used to going in directions that seem counter intuitive, but less mature ones really struggle with that one.”

Start With Full Audience Optimization

So what do you do? Start with full audience optimization (crawl first). Stephen Pavlovich from Conversion.com told me a while back that one of the bigger mistakes companies make is trying to copy sophisticated testing programs before they’re ready. As he said…

Stephen Pavlovich:

“Starting with full-audience optimisation first will mean that you’re optimising the experience for every user, not a segment.

It’ll also give you a strong understanding of how users behave and what they’ll respond to – which can then feed into personalisation. If you start personalising too soon, you risk limiting the results (by focusing on segments of traffic rather than the entire audience) and you risk forking the funnel before addressing the core conversion issues.”

From there, take a strategy like Andrew’s (discovery, exploit, measure, optimize/repeat) or employ an adaptive agent like Conductrics.

Complexity and The Ethics of Personalization

I haven’t really dove into the ethics of personalization, but that’s an issue, too.

You’re probably familiar with the example where Target predicted a teen girl’s pregnancy, sending her ads for maternal clothes before her father knew she was pregnant, and the cultural dialogue around personalization/ethics at that time. So there’s the creepiness factor of personalization.

Another issue is in specific relation to adaptive learning algorithms, where you may have innocent inputs but unethical or even illegal outputs. For example, your decision logic may target a segment of the population in which there are regulations in place to protect. How do you avoid this?

That’s another management problem you’ll have to deal with when you implement personalization. I know Conductrics does a great job coming up with algorithms that will produce targeting rules that are both machine as well as human consumable.

You don’t need to manually construct the targeting, but they make it very easy to look at the rules from a human perspective and maintain a level of control.

Conclusion

Personalization, of course, can be a wonderful technique. You can customize experiences, vastly improving user experience and ROI. But with personalization comes complexity in the form of assignment/decision logic, as well as management and optimization.

That’s not to say you shouldn’t dip your toes in personalization; it’s just you should have a rigorous and mature testing program in place first.

From there, follow the same rigor you use to optimize your site with testing.

Feature image source

Digital & Social Articles on Business 2 Community

(111)