I’m as guilty as my fellow bloggers in treating digital analytics as if it’s a different animal rather than an extension of market research that’s been around since the old days when women tested recipes by asking friends and neighbors to try it out.
Well, today we’re gonna fix that.
Market research vs digital analytics
Market research commonly refers to intentional data collection designed to provide guidance for marketing and operational decisions.
How is market research different from marketing research? Good question and you’ll get lots of answers. In academe, which is where I spend much of my career, we use the term marketing research to refer to research about marketing (ie. academic research) and market research as research done to support decision-making in firms.
Here’s the definition presented by Wikipedia:
Market research is a key factor in maintaining competitiveness over competitors. Market research provides important information to identify and analyze the market need, market size and competition. Market-research techniques encompass both qualitative techniques such as focus groups, in-depth interviews, and ethnography, as well as quantitative techniques such as customer surveys, and analysis of secondary data.
Market research, which includes social and opinion research, is the systematic gathering and interpretation of information about individuals or organizations using statistical and analytical methods and techniques of the applied social sciences to gain insight or support decision-making.
The definition provided by the AMA for marketing research goes more like this:
Marketing research is the function that links the consumer, customer, and public to the marketer through information–information used to identify and define marketing opportunities and problems; generate, refine, and evaluate marketing actions; monitor marketing performance; and improve understanding of marketing as a process. Marketing research specifies the information required to address these issues, designs the method for collecting information, manages and implements the data collection process, analyzes the results, and communicates the findings and their implications.
I personally like the AMA definition better, although it combines elements of market and marketing research. I like it because it’s broader — including collection of any type of information that aids decision-making as well as assessments of marketing processes. The choice of definitions is up to you and, for our discussion today, it really doesn’t matter whether we’re talking about market research or marketing research.
In contrast, digital analytics refers to:
Digital analytics is the current phrase for describing a set of business and technical activities that define, create, collect, verify, or transform digital data into reporting, research, analysis, optimizations, predictions, automations, and insights that create business value.
Meanwhile, Avanish Kaushik, Google’s analytics evangelist, defines digital analytics as:
Digital analytics is the analysis of qualitative and quantitative data from your business and the competition to drive a continual improvement of the online experience that your customers and potential customers have which translates to your desired outcomes (both online and offline)
Thus, digital analytics comprise a marketing information system, rather than a market research effort. Yet, the 2 are complimentary and both are necessary to achieve the aims of the business. Read the article above to learn how a marketing information system and market research compliment each other.
Doing good market research
Market research can answer many questions and reduce risk. Take a look at the infographic below from our friends at Survey Goo. They offer a variety of examples using market research and benefits resulting from good market research.
One aspect of this infographic is really important and that’s WHEN to do market research. Obviously, digital analytics and a marketing information system are ongoing efforts to use data to guide decision-making. As a point-in-time effort, market research requires some impetus to start.
Sometimes, that impetus is based on chronology. For instance, you might produce a particular report on a quarterly or annual basis. A good example is the annual survey of social media conducted by Social Media Examiner. In other cases, you might conduct a market study before launching a new product.
My main point here is that market research should precede organizational planning and should be conducted in a manner to generate usable results.
Let’s deal with these 2 problems separately.
When to do market research
The time to do market research is when your ideas are in a fairly nascent stage, before you’ve spent much money on them. Once you have a prototype or are ready to launch the product, it’s too late for market research. You need feedback from your target market early and often in the design process to answer the following questions:
- Who is your target market and what do they need?
- Does the product address a real problem faced by your target market?
- Do a sufficient number of people or organizations have the need, money, authority, and desire to solve the problem?
- What do competitors offer and how can you differentiate your product from theirs in a substantive way?
- What features should you offer? Which features are consumers willing to pay for?
- What should your product look like?
- What are the claims that will resonate with your target market?
- How much are members of your target market willing to pay?
- Where would they anticipate buying your product?
- What challenges would they face in buying and/or using your product?
Not only is market research important in the early stages of product development, periodic market research let’s you know where you stand, answering questions like:
- What share of the market do I have?
- Which marketing activities contribute most to my sales? Which contribute the least? Which produce the highest ROI?
- How are competitors marketing their brands?
- Are there changes in my target market or the population that offer opportunities or threats?
- What can I do to increase my market share?
Problems creating poor market research
We’ve all heard horror stories of market research gone wrong.
Like New Coke.
How do you avoid these problems?
Hiring an experienced market research agency or experienced in-house researchers is important. Otherwise, your market research does more to confirm biases than guide decision-making.
Here are some dangers to watch out for:
1. Wrong sample
In my experience, firms too often use a sample that’s convenient rather than one that’s going to give valid information. If you ask sales people what customers want, you’re going to get something very different from asking customers themselves.
So, your first task is to identify a valid sample. While I’m not convinced that a random sample is the end-all sampling technique, you need a good reason for not using it. For instance, I once did a study of buying pharmaceuticals on the Mexican-side of the US border. We stood on a bridge between the 2 countries and ask questions of every Nth person crossing the bridge. Sure, this wasn’t truly a random sample, but there’s no reason to believe that folks crossing other bridges or on different days were systematically different from the ones we talked to. Alternatively, selecting a random sample from a list would have been challenging since we wouldn’t find many who did this cross-border shopping.
Meanwhile, I heard a colleague touting her random sample until I pointed out that her demographics didn’t match Census data, suggesting bias in her sample.
Chief take-away: ask the right people and use an unbiased method of choosing members of the sample.
2. Too small a sample
Most of us remember taking Stats in H.S. or college. We hated the formula for calculating sample size because it relied on variance, which we didn’t know, in many cases, until we’d analyzed our sample. That said, I think we can all agree those polls in the newspaper are seriously flawed when they use a sample of 200 to project anything about the US with a population of over 300 million. It’s just too small.
But, do you need 20% of the population to be valid? Or 10%?
My guess is you need a minimum of 5-10% of a B2B population and about 20% of a consumer population. But, that’s just a guess.
Chief take-away: you need a sufficiently large sample of a population to capture the variability within the population.
3. Ask the right questions
I was taught to use multi-item scales for data collection. That might not always be possible, but too few practitioners even know about scaling or constructs or other issues related to how you ask questions.
By the same token, they ask double-barreled questions, questions that no real person could answer correctly, or questions that lead to a specific answer. Multi-item scales are designed to reduce these problems.
We’ve probably all come across surveys that fall into the category of “bad questions”. Maybe they ask about things you didn’t experience and don’t give you the option to not answer the question (this is more common in electronic surveys where the system is set to lock until all questions are answered).
In the Coke example from earlier, they asked about sweetness when that wasn’t a determining factor in deciding between Coke and Pepsi.
Chief take-away: ask the right questions in a way that provides valid, unbiased answers.
4. Do the right analysis
Planning for the right analysis starts when you’re crafting the questions. For instance, I once suggested to a client they needed conjoint analysis — a survey technique that provides guidance in terms of what features customers are willing to pay more for. They had already constructed the questionnaire so they didn’t want to go back and redo it. Once the data was analyzed, they asked me which features to put into the new device and I didn’t have the data to answer them.
Chief take-away: determine what questions you need answered, then construct the survey to provide that type of data.
5. Interpretation matters
The most critical element of a market research study is interpreting the data; what does it mean for our organization; how does it impact our decisions.
And, there’s no shortcut in this. You need someone intuitive, someone who understands your company and its needs, someone who knows how to use the data to its fullest advantage. For instance, I was once analyzing a large dataset. Some aspects didn’t fit my initial feelings about the situation. That lead me to dig deeper and I realized 1 variable hadn’t been recoded properly. Once we fixed that, the data told the story I’d expected after a series of qualitative preliminary studies.
Chief take-away: interpretation is part art and part science.
A few final caveats:
- Don’t assume a single study is all that’s necessary. Often, combining a qualitative and quantitative study provide superior results.
- Just because you have a hammer doesn’t mean that’s the right tool for every job. Sometimes you need a different tool to get at the right result.
- Unlike finance, market research is probabilistic. That means you’re getting results in terms of what’s most likely to happen, but it’s not always what WILL happen.
- Knowing something doesn’t mean it will be true from now until forever.
- Doing market research to support what you already know is bad market research.
(88)
