Using Virtual Surveys to Gather Project Data in the New Normal

Restrictions in response to the coronavirus disease (COVID-19) limited the conduct of traditional methods of data gathering. Photo credit: Bilguun Erdenebat.

Share on:           

Published:

Computer-assisted surveys make data collection possible despite COVID-19 travel and social restrictions.

Introduction

A virtual survey is a method for collecting primary data that uses a computer with various levels of interviewer involvement. It is time- and cost-efficient, reduces error during data transcription, enables the use of multimedia elements, and contributes to lowering the carbon dioxide (CO2) footprint associated with travelling for field-based data collection. While the use of virtual surveys is not new, lockdowns in response to the coronavirus disease (COVID-19) outbreak have highlighted their potential as one of the most viable options for data collection for development initiatives.  

This explainer is based on the lessons learned from using virtual surveys for conducting social and gender assessments in the People’s Republic of China (PRC) and Mongolia during the Great Lockdown of 2020.

What are the uses and types of virtual surveys?

Virtual surveys can be used at any point in the project cycle to overcome distance and travel restrictions. These surveys can be used to conduct due diligence including social and gender assessments, indigenous peoples census and household surveys, public consultations, and technical due diligence.

During the implementation stage, these can be used for monitoring or tracking project indicators, and can also be useful for conducting project evaluation and beneficiary satisfaction surveys.

Virtual surveys include:

Computer-assisted telephone interviews (CATI)
An interviewer communicates with respondents remotely and enters their responses in a computerized questionnaire.

Computer-assisted web interviews (CAWI)
Data is collected via online surveys through a web interface. Respondents answer the questions on their own computer, tablet, or smartphone.

Market options for web-based surveys include Google Forms, Survey Monkey, Typeform, and SurveyGizmo. These require internet access. Survey Solution is a platform that can work offline, but it requires mobile equipment and a survey team on site. For surveys conducted in the PRC, Wenjuanxing and QQwenjuan are the go-to virtual platforms.

Computer-assisted personal interviewing (CAPI)
Interviewers collect data using tablets or mobile phones; data is transferred to a survey management website.

Social media surveys
Data is collected through platforms such as Facebook, LinkedIn, Twitter, and in the PRC, WeChat.

Virtual focus-group discussion
Video conferencing and chat platforms are used to bring together participants for a structured discussion. Platforms include Zoom, Google Meet, Skype, and MS Teams. In the PRC, QQ and WeChat groups are the popular choices.

 

Text message survey
Responders receive polls, voting opportunities, or links to survey sites through text messages (SMS). SMS is a good choice as mobile use is growing steadily, even in rural areas. The downside is that an SMS survey is usually one-way. Platforms such as FrontlineSMS make 2-way interaction possible, including in extremely poor areas without internet

Mobile applications
Responders download a custom-made application and are provided with log-in credentials to access the survey.

Mixed method virtual survey
Data is collected using a variety of virtual survey methods or a combination of virtual and face-to-face methods.
What are the virtual survey’s advantages over conventional methods?
  • Recruitment and replacement of survey participants is efficient and effective.
  • Web-based control platforms enable immediate data export and real-time data availability as the survey is underway.
  • On-the-spot data input increases data quality. Data gaps or unclear statements can be identified and clarified immediately with respondents.
  • Virtual surveys can capture location through a global positioning system (GPS), time stamps, and geographic information system (GIS)-integrated data.
  • Virtual surveys allow for a system of records accessible and referenceable throughout the project life and enable data analytics for progress monitoring.
  • They are cheaper, reduce or eliminate travel and extensive data entry costs, and can be conducted in less time than conventional ones.
  • They also reduce CO2 emissions linked to travel and lessen the use of paper and printed materials.
What are its disadvantages?
  • People without mobile phones, with poor mobile network access, or who have low digital literacy may be left out in virtual surveys leading to sampling bias.
  • It is difficult to provide incentives for respondents and this can reduce interest.
  • Participants can incur costs for mobile or internet access charges when answering the survey or downloading survey forms or apps.
What are the steps in conducting a virtual survey?

  • Define the objectives of the survey.
  • Identify the target groups, types of questionnaires needed for the various respondent segments, sample size, and geographical distribution.
  • Develop the electronic survey questionnaire.
  • Prepare the survey budget. Consider the fees for the use of the software or digital platform, purchase or rental of handheld devices and wireless routers, developer fees for app development and web control system, survey team recruitment and related costs, local travel cost if CAPI methods will be used, tokens of appreciation or mobile data bundles for respondents, and other unforeseen expenses.

  • Consider hiring a local survey team if you are using CAPI or mixed methods. Train them remotely and establish a virtual social media network with them for continued communication during survey implementation and data processing.
  • CATI methods have proven effective for recruiting participants and registering demographic information without distance restrictions. Consider partnering with private survey firms or local organizations to get support in accessing local datasets of potential respondents.
  • Establish a communication and feedback mechanism with the field enumerators to solve technical and methodological problems during data collection. Before the survey, provide a detailed description of the survey principles and requirements.

  • Consider software that is commonly used in the project area and that is easy to use, download, and install.
  • Note that the free version of online survey platforms tends to be limited. Local survey firms usually have access to the professional versions.
  • If your project is in the PRC, Wenjuanxing and QQwenjuan are good online platform choices.
  • If you have enough budget and time, you could develop an exclusive data collection tool, such as a dedicated app. This will give you flexibility to align your survey to local language and culture, avoid confidentiality issues, and encourage participation in areas with less acceptance of a third-party survey software.

  • Make the questionnaire as simple as possible. This should be tested prior to implementation.
  • Add a detailed description of the project information at the beginning of the questionnaire so the respondents will have a solid understanding of the objectives of the survey.
  • Conduct focus group discussions and interviews with key informants to improve the design of the questionnaires.

  • Train enumerators in the field on how to distribute the electronic survey and how to fill it out for people who do not have their own mobile devices.
  • Provide tokens of appreciation to the respondents to encourage their participation, especially among the poor and if they incur mobile or data charges.
  • Keep the contact information of respondents, such as telephone number, social media account or email for further interview or clarifications. Note that this must be handled with special care. Researchers must follow ethical standards. Contact information should only be used for survey purposes.

Complement the assessment with literature review and triangulate with relevant studies, if available.

Establish a private social media group for interested participants for continuous engagement and communication. This can be useful for longitudinal surveys, or to assess the progress and impacts of ongoing project operations. Such groups can be called “online interest group communities” that participate during the whole lifecycle of the project, from design to implementation to evaluation. The members can share their experiences on the changes brought about by the project. Project managers can also obtain feedback and suggestions from the community on an ongoing basis. Social media groups have to be managed with extreme caution, follow ethical standards on confidentiality, and limit their use for survey purposes.

Sample Cases:

Case 1: Using virtual surveys to prepare a new project in PRC

Case 2: Capturing time-use during the COVID-19 pandemic in Mongolia

Recommendations
  1. Use mix methods (virtual and face-to-face or paper-based) to increase the participation of people with low levels of digital literacy or limited access to smartphones or mobile network.
  2. Keep it simple. Your questionnaire should be short with no more than 30 questions. If using mobile apps, these must be straightforward and intuitive.
  3. Instructions for respondents must be clear to overcome the challenges of limited face-to-face interaction. You can enhance communication with respondents to solve issues and answer questions during the survey using chat platforms, SMS, or app-based notifications.
  4. Good quality panel data can save time and costs of recruitment work. Consider collaborating with community organizations that could reach target groups through their social networks or work with local research companies with access to large datasets. Consider sample size and crosscheck survey results against other findings.
  5. Invest in training your survey team. This will help ensure smooth interaction with respondents and increase efficiency. Ensure the highest ethical standards for privacy and interaction with participants.
  6. If possible, provide tokens of appreciation to the respondents for their time and to increase their interest in participating, particularly if you are surveying low-income participants. Consider mobile data credits to cover data usage incurred by the participants for undertaking the survey or negotiate with mobile operator companies to reduce or remove such cost from the phone bill of participants.
  7. Plan enough time for the development of virtual tools. Especially if you are new to virtual surveys, take the time to ensure the tools are user-friendly and adequate.

Veronica Mendizabal Joffre
Senior Gender and Social Development Specialist, Gender Equality Division, Climate Change and Sustainable Development Department, Asian Development Bank

Veronica Mendizabal Joffre leads ADB’s gender equality program for Southeast Asia and has previously managed the gender program in the People’s Republic of China and Mongolia. With a background in public policy and knowledge management, she has been working on development issues for the past 17 years. Her research interests include labor force dynamics in the context of technological, economic, and demographic change, and the gender dimension of technological transformation.  

Huijuan (Emily) Wang
Social Scientist

Huijuan has over 14 years of professional experience in social and gender mainstreaming and safeguards. She has worked with various multilateral development banks in the region and has prepared over 50 loans. She has a PhD from Hohai University in the People’s Republic of China.

Zhou Jian
Social Development Specialist (Safeguards), Office of Safeguards, Asian Development Bank

Zhou Jian has more than 15 years of professional experience in resettlement and social impact assessment. Prior to ADB, he was a researcher in the National Research Center for Resettlement of the People’s Republic of China. He also worked as a consultant for World Bank, ADB, EIB, and USAID.

Purevtsengel Luvsandandar
Research Manager, Mongolian Marketing Consulting Group (MMCG)

Purevtsengel is a social researcher dedicated to the development of Mongolia. Her recent studies include socioeconomic aspects of public transportation services in Ulaanbaatar, a rapid assessment of time use during the COVID-19 pandemic, and a rapid assessment of COVID-19 impacts on the food and agriculture sector of Mongolia.

Narantsetseg Jaalaa
Researcher, Mongolian Marketing Consulting Group (MMCG)

Narantsetseg has 13 years of experience in media monitoring and research. Her recent projects include national election polls and child protection in cyberspace in Mongolia. She has a degree in Policy Analysis.

Asian Development Bank (ADB)

The Asian Development Bank is committed to achieving a prosperous, inclusive, resilient, and sustainable Asia and the Pacific, while sustaining its efforts to eradicate extreme poverty. Established in 1966, it is owned by 68 members—49 from the region. Its main instruments for helping its developing member countries are policy dialogue, loans, equity investments, guarantees, grants, and technical assistance.

Follow Asian Development Bank (ADB) on
Leave your question or comment in the section below:
Disclaimer

The views expressed on this website are those of the authors and do not necessarily reflect the views and policies of the Asian Development Bank (ADB) or its Board of Governors or the governments they represent. ADB does not guarantee the accuracy of the data included in this publication and accepts no responsibility for any consequence of their use. By making any designation of or reference to a particular territory or geographic area, or by using the term “country” in this document, ADB does not intend to make any judgments as to the legal or other status of any territory or area.