There is no denying that marketing research is evolving. New technologies, the abundance of data available and consumer dependence of smartphones being the biggest contributors. But how does this change the way we sample via online research? And source permission-based, quality respondents?
Austen Lear, Sourcing Manager at Kantar's Profiles Division, answers questions on sampling and sourcing for modern surveys, best practices and common mistakes.
Q: A lot has changed in online market research over the past decade, and much of that revolves around how we sample and source respondent data. What are the key opportunities or challenges you see regarding sampling and sourcing today and into the future?
I would say that technological enhancements have provided a huge opportunity in terms of how the industry have approached the respondent supply chain to the survey. Kantar has embraced this in terms of the options to deliver respondents to surveys across a wide network of compliant sources. We have now integrated multiple sample sources into our platform which means we can more quickly access about 80 million respondents across the globe.
As for challenges, from an industry view point, I believe it is important for clients to understand more from their suppliers about where their respondents come from and how they are managed. The enhancements in technology have probably led to less transparency on sourcing and sampling being used. At Kantar, we continue to run source comparison surveys and the differences between how sources differ in terms of results is quite marked. Having an understanding of this is important to help support the client in terms of the sourcing and sampling options to support their research design.
The sampling step always needs a level of consideration about who you want in the research population, and how sources can accommodate that. Understanding and discussing the research design step and the budget is ultimately the most crucial step in assessing the sampling method options.
I am also interested to see how automated platforms pan out in the next 3-5 years. Kantar partnered with Cint in 2018 to upgrade our fieldwork management, which in turn updated our sourcing and sampling methods. Our partnership with Cint sets us apart from most as we obtained a tool that is already proven. It will be interesting to see how the boom of automated sourcing and sampling impacts the industry down the road.
Q: What are the most frequent questions you get from clients?
The top question I receive stems from clients who become worried about adding or taking away a source to their tracker that may date back several years. Clients are always asking how a particular source will change the way the data looks for their research. In past years this has been somewhat difficult to answer, but this year our Global Sourcing team has conducted a side by side test of many of our most used vendors, starting in the US and will broaden to other key countries. This will allow our teams to make informed and educated decisions when creating or altering a sample blend for a client. I am very excited about this information and how it will facilitate great conversations with clients.
Q: Is “river sampling” a thing of the past?
Yes, the phrase “river sampling” is somewhat a thing of the past. Most sources that one considered as ‘river’ are in fact dynamically controlled flow from a set of suppliers. There is probably room in the industry for a true ‘river’ solution.
In 2019, we are launching an updated Approved Supplier List that will help us improve sourcing decisions utilising knowledge on the network of compliant sources including the type of source they are or offer. This will help the decision making as to when to use a particular source to support, where applicable, Kantar’s proprietary research panel, LifePoints.
Q: When sourcing respondents for global online research, what are three considerations every researcher should keep in mind before fielding?
Understand the population to be covered and how online can cover those populations adequately. Again, sometimes there are trade-offs between coverage and client budget. What I mean by that is this, all facets of the study need to be double checked to ensure there won’t be a hurdle within a specific country. These could be as simple as modifying demographic quotas from country to country. Some demographics – like lower social classes in developing countries are not adequately covered online and through the sources available today. It could be asking about a certain brand that isn’t prevalent or doesn’t exist internationally. Certain questions can be frowned upon internationally, whether that’s income, ethnicity, etc..
Use trusted and compliant sample sources. Our Approved Suppler List will help everyone through this process. It’s important to know what sample sources are available before beginning fieldwork to ensure the sources are vetted and trusted in terms of compliance, experience and quality.
Consider whether the survey is designed to be taken on desktop only or for all devices including smartphones. Mobile is used far more for many of our lesser developed countries. Making sure our surveys are mobile friendly are always top of mind, but we should understand how a particular country responds to our surveys. This can impact demographic quotas in addition to answering behaviour and needs to be factored into the data.
Q: What’s the most common sampling mistake you experienced in 2018, and what can researchers do to avoid it?
Overcomplicating quotas. We need to discuss the approach of each study with our clients. By taking a look at the goal of the research we may be able to reduce the amount of quotas needed. A few things to keep in mind…
1. "Region" will fall out on its own most of the time.
2. Age breaks can be broadened, and there is rarely a need for 6 age breaks: 3 to 4 should provide the necessary data you’re looking for, but it should be discussed.
3. Quotas can be monitored during field, and we can bend and flex with the study utilizing our new sampling platform much easier than before. It could be best to set up a simplistic quota structure and if you notice a certain quota bucket isn’t filling as much as you’d like, then alter the target groups to help that quota group along.
