Beware the Changing Landscape of Market Research

Merrill Shugoll
April 27, 2016



The following post is republished from Pratt Street Communications' Pratt Street Stories

In this era of online surveys and social media analytics, the very definition of “market research” has changed. While these electronic tools offer fast data, the information they yield comes with some cautionary fine print.

Traditionally, market research is defined by the American Marketing Association as the systematic and objective identification, collection, analysis, and dissemination of information for the purpose of improving decision-making related to the identification and solution of problems and opportunities in marketing.

Today, however, market research has become synonymous with online surveys, social media analytics, and, the proverbial, asking a few people what they think. But are these results reliable and projectable to the population under study, so that sound business decisions can be made with confidence? Perhaps more importantly, are these results being used to the exclusion of more robust sample sizes, appropriate methodologies, and questionnaire integrity so that important business opportunities are revealed?

Of course, you would expect a classically trained and experienced researcher like me to promote the value of hiring a professional research firm instead of doing it yourself. But I’m not saying this out of self-interest. I’m saying it because I’ve devoted my entire career to using strict methodology to ask the right questions of the right audiences, to find the right answers. And I’m worried that people conducting their own surveys online or using social media or gathering opinions through informal conversations aren’t getting truly accurate results, for several reasons.

First, people pursuing their own research often don’t know what methodologies are appropriate to use for answering specific types of questions. For example, it is not uncommon for a “DIY” researcher to think he or she can get answers to product or service pricing questions by using qualitative research. Conversations about price and value should not be the basis of pricing decisions. The reality is that quantitative research is much better suited for these type of questions. A larger sample is needed, and, furthermore, the questions need to be phrased precisely so that the findings are projectable.

Another challenge is that non-research professionals tend not to be familiar with questionnaire design techniques and sampling methods. This often leads, inadvertently, to unbalanced scales or overlapping answer categories. As a result, study findings can actually be very misleading. Questions may include bias, and they may, potentially, be directed at the wrong people.

Social Media Research (SMR), is another area where data gathering and analysis is not as clear-cut as it seems. According to the Marketing Research Association’s Social Media Research Guide, “Social Media Research (SMR) covers all research activities where the information being used is derived from the social media space” (e.g., Facebook, Twitter, blogs, Google+ and other channels).

Information is collected quickly and relatively inexpensively, but little consideration is given to the biases inherent in the sample. In other words, who exactly is responding? What is known about them? How much are you really learning?

  • It’s important to keep in mind that using social media is not always feasible to research local or small brands/niche product categories because there is rarely enough chatter available to analyze.
  • Additionally, only public posts can be observed, which limits the amount of data available. (Facebook, of course, is mostly private.)
  • Also, researchers have little control over the discussions that ensue on social media channels, so information objectives may or may not be met.
  • Finally, sentiment scoring is mostly on a positive-negative scale, which limits analysis of language that conveys more complicated feelings.

What is the answer to these challenges? As president of a national research firm that’s been in the business for 60 years, I think the answer is a robust audience sample and a combination of methodologies (i.e., hybrid approaches) to ensure delivery of reliable data. Reliable data gives the researcher the ability to draw conclusions and make actionable recommendations. After all, isn’t that why you’re doing research in the first place?

Share this page

More from Shugoll Research:

Next Post:
Five Things To Do When Visiting National Harbor

Previous Post:
Announcing Our Mobile Streaming Service