How Mobile Technology Enables Better SDGs Monitoring
Using a tablet or smartphone to conduct survey interviews makes it easier and faster to benchmark progress toward the Sustainable Development Goals.
Achieving the Sustainable Development Goals (SDGs) depends on accurate, timely, consistent, and comparable data. Countries need to keep track of progress toward 169 targets and 232 performance indicators. This is an arduous task especially for national statistics offices (NSOs), which play a key role in collecting, validating, and reporting data. Several indicators rely on census and survey data that are costly to collect and take a long time to process.
Fortunately, the increased computing power and affordability of handheld digital devices, such as tablets and smartphones, offer a cost-effective solution to gathering field data and generating timely and accurate statistics. As a result, more survey practitioners are using computer-assisted personal interviewing, also known as CAPI.
A report from the Asian Development Bank (ADB) provides evidence on why CAPI is a better alternative to the traditional pen and paper interviewing (PAPI) method and offers lessons from pilot implementation in Sri Lanka and Viet Nam.
What is CAPI?
CAPI refers to the use of computers, particularly handheld digital devices like tablets and smartphones, in data collection and survey management. It can help enhance statistical capability by providing timely, easily accessible, and high-quality data.
Software platforms for CAPI provide built-in data checks, navigation tools, easy data transfer options over the internet or via Bluetooth technology, and the ability to capture global positioning system (GPS) coordinates, photos, and other relevant information about the survey implementation.
Who should use it and why?
Developed economies have been using CAPI for several years. NSOs in developing economies, however, have yet to take full advantage of this method because of high startup costs and lack of technical capacity to develop and implement CAPI surveys. They rely almost exclusively on the PAPI method for data collection. This is a tried and tested technique, but it faces several problems in delivering the data required for the monitoring of the SDGs. These include a large amount of paper used by PAPI, increasingly complex surveys, timeliness and accuracy of data, and high cost of surveys particularly for economies with large populations.
Benefits of CAPI
CAPI offers a good alternative to this traditional method. Here are some of its benefits:
- Eliminates the need for paper questionnaires. A mobile device replaces the large volume of survey forms that census takers or enumerators carry from house to house. Completed survey questionnaires can be electronically transmitted after completion from the field to the survey manager.
- Saves time since survey responses are directly recorded on a tablet or smartphone, eliminating the need for data entry on a computer at a later stage. This allows for quicker access to field data for review and analysis.
- Ensures the quality of data since errors are likely to be reduced because of the validation checks programmed into digital questionnaires like automated skipping of questions that are not applicable to the respondent.
- It can capture photos, record audio, store GPS information, and provide other features made possible by digital technology.
Evidence from the field
Results from ADB’s technical assistance project show that switching to CAPI can shorten the overall time to produce a clean dataset and improve data quality significantly.
In Viet Nam, for example, the study found that shifting to CAPI from PAPI reduces interview duration by an average of 9.42 minutes, which is 27.3% less time spent interviewing a household.
CAPI lowered the number of errors by 0.8 per survey interview in Viet Nam and by 6.2 per survey interview in Sri Lanka. To put this into context, for a survey of 10,000 respondents, the findings translate to a reduction in total errors of 8,000 for Viet Nam and 62,000 for Sri Lanka.
CAPI’s fixed costs—generally associated with programming the software—might initially be higher relative to PAPI. However, since CAPI virtually eliminates PAPI’s variable costs, such as printing questionnaires and data entry, it becomes cheaper to implement as sample size increases. The breakeven point—the sample size at which it becomes cheaper to implement CAPI over PAPI—was calculated to be 1,769 households for Viet Nam and 1,467 households for Sri Lanka.
How does it work?
Figure 2: Data Flow
CAPI involves fewer steps than PAPI since it does not require printing or data encoding.
The first step is to create and design the questionnaire on a CAPI platform.
Next, test the digital questionnaire to make sure that all functionalities (e.g., embedded data checks) are working.
Then the head office assigns supervisors and interviewers to a specific area to conduct the survey using a mobile device. The interviewers submit the completed form to the supervisor using the same device.
The supervisor has an option to use a desktop computer or mobile device in checking the questionnaires on a computer and has the option to accept or reject a form and ask the interviewer to verify some data. The interviewer submits the form again after verification.
The supervisor transmits the forms to the head office, which also has the option to accept or reject the form and require further validation.
The process flow can also be modified on many CAPI platforms based on the design and implementation of the survey.
What are the best practices and lessons from the field?
The CAPI questionnaire design will affect the overall survey and post-survey activities, such as data cleaning, validation, and processing. That is why designing and programming the questionnaire properly is important.
CAPI should be seen as having the same function as a data entry system in a traditional paper-based survey. The CAPI system is programmed to not only match the developed questionnaire but also to serve as a platform for recording interview-derived data.
Testing the survey before field implementation ensures that the digital questionnaire is working properly is of utmost importance. The system can still be adjusted and revised based on test results to produce an error-free CAPI system.
Training of enumerators on the use of CAPI is vital in the successful implementation of the CAPI survey. Interviewers should know the objective of the survey and how to use the system (e.g., familiarity with the questions and functionalities of the platform, proper inputting of the data properly).
ADB has partnered with the Food and Agriculture Organization of the United Nations to develop free online courses on CAPI using CSPro and Survey Solutions accessible through the internet. These online courses provided practical training to statisticians and NSO employees around the world, guiding them on how to implement a computer-assisted survey using software that they already use. A second round will be implemented in the second quarter of 2020, and those interested in taking the course can register online.
A solid backup system should be in place in case of technical failure. This could be achieved by not just maintaining a copy of the data on the tablet but also a dedicated computer server or cloud storage and backup. Another option is for interviewers to use Bluetooth to back up files to a colleague’s tablet or their supervisor’s laptop when there is poor or no internet connection.
Enumerators will be transferring and backing up confidential data about identifiable respondents through the internet, which needs utmost care in data handling. Security protocols (i.e., who should have access to the data) should be in place. Where to store the data also matters. Some national statistics laws provide that data should reside within the geographic boundaries of a country. This means using a cloud server is not an option.
It is important to keep track of ancillary data on the interviews and the survey process, which is commonly referred to as “paradata.” Information, such as how long the interviews took, were records used to answer survey questions, which enumerator conducted the interview, the number of errors, GPS location, tablet, and questionnaire versions, are all useful data that can help plan resources during the survey and for similar surveys.
ADB’s experience shows that CAPI was well accepted in urban areas, where people are more exposed to technology. However, the opposite is true in rural areas. Outreach and education campaigns on CAPI may be necessary prior to the launch of any large-scale survey to ensure full cooperation of survey respondents.
The CAPI Effect: Boosting Survey Data through Mobile Technology. A Special Supplement of the Key Indicators for Asia and the Pacific 2019.
Leave your question or comment in the section below:
YOU MIGHT ALSO LIKE
The views expressed on this website are those of the authors and do not necessarily reflect the views and policies of the Asian Development Bank (ADB) or its Board of Governors or the governments they represent. ADB does not guarantee the accuracy of the data included in this publication and accepts no responsibility for any consequence of their use. By making any designation of or reference to a particular territory or geographic area, or by using the term “country” in this document, ADB does not intend to make any judgments as to the legal or other status of any territory or area.