Take, for example, an insights professional planning a survey of 1,000 people who drive a specific make and model of motorbike and live in the London area. A traditional survey approach would be to send the survey to 200 people who have been pre-identified by a research panel and ask many more about their ownership of motorbikes in order to try to get the 1,000 motorbike owners. While this is a longstanding practice, it runs contrary to achieving speed to insight, not to mention that it does not improve upon the survey experience.
The good news for market researchers who face this problem is that help is on the way.
Machine learning technology is emerging that will help online survey tools predict the answers to questions like “Do you own a motorbike?” The technology teaches computers to learn from experience – learning from data without relying on a pre-determined equation as a model. Machine learning algorithms adaptively improve performance as available sample numbers for learning increase.
Machine learning has an expanding presence in marketing, such as in online advertising as a tool for making digital campaigns more targeted and personalised. Another way marketers leverage machine learning is in customer experience applications to identify patterns among customer interactions to increase revenue opportunities. Market research is a natural extension of this technology trend.
With machine learning, survey platforms can learn and predict properties of users based on their answers to other questions and on demographic and profile data similarities to additional panelists. The technology enables insights professionals to rely less on asking questions to qualify respondents, which means less wasted time and resources. It aligns with respondents’ expectations that researchers already have knowledge about them and don’t need to ask basic information questions which can turn them off.
To return to our motorbike example, machine learning will enable surveys to primarily target people who are likely to have a high chance of owning a motorbike, even though they have never specifically answered the question. Based on this learned intelligence, a survey will simply ask respondents to verify this fact as they enter the survey, reducing the number of people rejected and boosting panel satisfaction because the survey is attuned to their profile.
Machine learning addresses a challenge that is impossible to handle with traditional heuristic-based approaches; these methods would not perform well and would be impossible to scale. Survey panels can run into millions of users, with each panelist having thousands of data points – potentially billions of data points in total. Machine learning techniques address this scale issue to learn about panel users based on their activities and then predict their answers to questions.
In examining the ways in which machine learning can advance market research, Toluna has used an open-source library developed from Google to investigate and compare a number of learning algorithms. Our research shows that this technology can help improve targeting significantly. Survey design will continue to first look for people who match the target and then look for people whose predicted answers match the target. We expect this new process will reduce panel fatigue significantly. It will also help drive more completes per survey panel and require less capital expenditure for each survey.
Machine learning technology shows great promise in market research. It can learn from huge amounts of data to generate insights and predict answers without asking irrelevant questions to panelists, improving respondent experience and survey results.