When should we use online research instead of offline methodology?

When you think about the different types of research data collection methods that are available, what comes to mind? Do you think about the speed and lower cost of using telephone methodology? Or do you think about the data accuracy and the flexibility in terms of subject matter when using an online methodology?

Market Research at heart is “asking the right people the right questions and understanding their answers”. Having a good understanding of the target audience and the different ways of reaching the right people is undeniably an important part of the process of achieving the desired objectives and information.

So what are the things we should consider when deciding on the right data collection method to use? Researchers have theoretical and practical considerations when choosing the best method of fieldwork.

1. Theoretical considerations

a. Coverage or the best possible representation of the population being studied: Are the sources from which the sample is drawn reliable and of high quality?

b. Accuracy of the Data: Does the data truly represent what the respondent said? Is it truly what the respondent thinks?

 

2. Practical considerations

a. Speed to Analysis: How quickly do we need the data?

b. Cost: How much money do we have to spend to get the desired information?

c. Subject matter: Does it require us to discuss some personal topics? Do we need to show actual products or any communication aids?

d. Interview length: How engaged will the person be when they’re taking the particular type of survey?

????s

In a recent Global Market Research Report by ESOMAR, online research steadfastly stood as the most popular data collection methodology, followed by the more traditional types of research, telephone and face to face.

However, this is not always the case if we study certain countries separately. In Singapore, Japan, Australia, and the UK, online quantitative tests take precedence, while face-to-face interviewing is still more popular in China and Brazil. Telephone interviewing, on the other hand, is the most prdontdontevalent type of fieldwork in Malaysia and Germany, based on global research expenditure.

The diversity in data collection methods across countries and even within a market simply shows the many different ways that researchers can engage the right people to answer those right questions. And so to help us identify the best method to use, we need to examine how well each of these methods racks up on the different factors under our consideration.

What is offline methodology, and what are its pros and cons?

Offline methodology refers to sampling data collection that is not done through the Web. The primary means of offline research is by telephone but can also include postal and face-to-face methods, including in-home, door-to-door, and hall/mall interviewing.

Here is more information on the main ways in which the data is collected for quantitative research:

Theoretically, postal has really good coverage, because it includes everybody except people who do not have a home address. However, in practical terms it is imperfect, because the source used for the addresses may be out of date or incomplete. This methodology also provides good data accuracy and flexibility in terms of subject matter and survey length, due to its self-completion format. But it generally takes longer and costs more, so to save time and minimize expenses, researchers sometimes conduct very short surveys via this method.

Face-to-face. In theory, it is very good. It can cover almost 100% of the target population by sampling in every street or location. But in practice, it is expensive and time consuming, especially when interviewing too many people in the same location. There is also potential for interviewer error, and bias or acquiesce, where the interviewee is just saying what they think the interviewer wants to hear.

Hall/mall tests. While coverage is imperfect even in theory (since the sample will be skewed towards those people who are at the location at a particular time), this method can save time and costs, provide good accuracy and flexibility in terms of subject matter and length if a computer-aided method (CAPI or Computer Assisted Personal Interviewing) is used and participants are allowed to input the answers themselves.

Door-to-door. Enables good-quality answers for a longer period of time, especially for those surveys where we need to assess people’s reactions to actual products or other types of visual media.

Telephone. Enables the interviewer to guide the respondent, speeds up the process, and reduces cost compared to some of the other methodologies. However, there are some practicalities that it also needs to contend with, such as working with harder-to-reach demographics (younger males) and very large samples that can be slow in the field; potential interview error and bias; shorter interview time, especially for mobile devices (usually lasts for 30 mins or less); inability to work with subject matter that requires visuals; and the use of directory lists that exclude some parts of the population (i.e. non-phone households). So in order to get good coverage and increase response rate, we need to include mobile-only households and work with call centres or interviewers that can establish the right rapport with the participants.

What is online methodology, and what are its pros and cons?

Technology and the prominence of the Internet have definitely given us more flexibility and creativity to enhance research design, to make connections with millions of people, and to communicate with them much faster. So it is not surprising to see that online (sampling data collection done through the Web) has now become a major focus of market research industry.

Let’s take a closer look at how this methodology fares, with our list of practical and theoretical considerations.

Cost. It costs less because of the absence of interviewers and data encoders. Much of the work can be automated.

Speed to analysis. Online is generally faster than all other data collection methods, especially for large sample sizes.

Subject matter. There is also a good level of flexibility in terms of subject matter, due to its self-completion format.

Length. It is generally shorter, because the respondents usually save time by reading and answering the questions quickly by themselves. However, there can also be a disengagement effect due to people’s level of inattention. There is no interviewer to help the person along, so the length of the survey is generally going to be shorter, and the quality of the answers lower than some of the other methodologies we’ve seen.

Accuracy. Just as with the postal methodology, we’re dealing with a self-completion format. Answers given will actually be what the respondent thinks or says, so there is a high level of accuracy in online methodology.

Coverage. It is imperfect, because not everyone has online access. And there is no universal sampling frame for us to draw on. In practical terms, we have to use access panels, and we have bias issues due to recruitment. The challenge is to ensure enough varied and broad sources in the recruitment to mitigate bias.

Should it be offline or online methodology?

That depends on the targeted country or population which our clients are looking for.

If we need to sample the entire country of China or Brazil, for instance, we need to use a more traditional method such as face to face or telephone, as not everyone may be reached through modern techniques such as online data collection.

However, bringing in some local understanding of the market, we know that more potential consumers, those who have more purchasing power in China, live in urban areas where there is high Internet penetration of about 70%. And this is the more appropriate segment to target for most companies in commercial products and services. Hence, online research can still be very useful in this country.

While not all countries have high overall Internet penetration, online can still be a viable solution in countries such as China, Brazil, and India, depending on the market that we are looking to study. We just need to keep in mind that such a sample may be skewed towards certain demographics such as young, more educated, and higher socio-economic classes or income groups.

So when researchers are thinking about the samples for their study, they need to take several factors into account in order to ensure that the data is representative of their target population.

And while each of the methods has its own strengths and limitations, this does not mean we have to stay within the boundaries that one method presents; researchers can utilize and combine some of the above methods as they cultivate their creativity and embrace new thinking in answering the important question: How can we conduct a survey more accurately, quickly, and inexpensively?