Designing Online Courses for Statistical Capacity Building

To leverage the numerous benefits associated with MOOCs for statistics capacity building, ADB organized two MOOCs in partnership with the Food and Agricultural Organization of the United Nations (FAO). The courses were targeted at enhancing the capacity of NSOs on tablet-based survey data collection, also known as computer-assisted personal interviewing (CAPI), but they were also open to anyone in the world interested in learning about this method. CAPI offers a cost-effective solution for gathering field data and generating timely and accurate statistics.

The aim of the MOOCs, like in-class training, was to equip learners with a solid understanding of how to develop and implement CAPI surveys using two popular platforms: CSPro for Android and Survey Solutions. The two courses ran over an 8-week period from 28 January to 22 March 2019 and were organized into eight sequential learning modules that were released on a weekly basis.

Each module contained lectures, as well as exercises and quizzes, that formed the basis for assessing learner performance. By the end of the course, learners were expected to at the very least grasp the fundamentals to develop CAPI questionnaires of their own. An optional capstone assignment was also offered at the end of the course to tests learners’ skills to program a sample questionnaire. We incorporated analytics to track learner progress and completion rates.

One of the criticisms of MOOCs is that course participation is almost entirely in the world’s most affluent countries. This is attributed to costs associated with getting official certification for having completed a course. Also, internet access can be problematic in some parts of the world.

Another criticism of MOOCs lies in its low completion rates—hovering around 3% to 6%. This is likely because course developers may not be creating the right incentives to sustain the interest of learners over time.

To address these concerns, we made the courses free to all participants and permitted downloading of videos and course transcripts. We also incentivized learners by offering an all-expense-paid trip to the top four- performers in the capstone assignment to participate in a regional workshop on CAPI that was organized by ADB in Bangkok, Thailand.

The course was marketed through numerous channels, including a crisp promotional video, brochures, and flyers disseminated on social media. We also reached out to other multilateral agencies to spread the word. These efforts led to a total enrollment of 655 learners, which is unusually high for a niche topic like statistical capacity building.

Nearly three-fourths of the enrollees came from Asia and the Pacific, while a fifth were from Africa. NSO staff accounted for more than a quarter of the enrollees, while other government staff accounted for nearly a fifth of the sample. A third of the 655 learners enrolled only in the CSPro course, while a fifth enrolled only in the Survey Solutions course, and nearly half of learners registered for both courses.

Figure 1: Learners' Country of Origin

In terms of completion rates, 34.7% completed the CSPro course, while 36.6% finished Survey Solutions; that is nearly 7 times the average completion rates for MOOCs! A 100% completion rate was observed for Module 1, but the biggest drop was observed in Module 2, with 63.1% and 60.4% of the learners completing the Survey Solutions and CSPro courses, respectively. This can be attributed to the slight increase in complexity by Module 2. The next largest drop was seen between Modules 2 and 3, with completion rates at nearly 50% for both courses. Learner engagement data also showed that for both courses, those who made it through at least Module 4 were most likely to finish the entire course.

Figure 2: Completion Rates Over Time

Source: Asian Development Bank staff estimates.

The four students who attained the highest overall scores for both the modules and the capstone assignment were sent to the workshop in Bangkok on 25 November 2019. Two winners were selected from each course: May Myint Bo from Myanmar and Christian Mina from the Philippines for CSPro and Roshini Abeyesekera from Sri Lanka and Samuel Nogbou from Ivory Coast for Survey Solutions. The winners presented their projects and provided valuable feedback from their respective online course experiences. 

The winners with course instructor, Lachlan Bruce (center), from left: Christian Mina, May Myint Bo, Roshini Abeyesekera, and Samuel Nogbou.

Based on student feedback from a survey administered at the end of the course, the MOOC platform was well-received and judged to be an effective tool for training a new generation of statisticians on CAPI.

The experience also generated a few lessons and recommendations moving forward.

First, ADB’s experience showed that MOOCs can reach a wider audience at substantially lower per-capita costs than in-class statistics capacity building activities. Further, strategic partnerships with other agencies maximized the use of resources in developing MOOCs on topics of mutual interest and widened the network of contacts to reach more people. Correspondingly, the wider reach of the MOOCs also enabled it to cater to typically underserved groups, such as women as well as individuals from low to lower-middle-income economies.

Second, our course was offered only in English, but the opportunity is seen in expanding the course offering to other languages, which would open doors to higher enrollments from non-native English speakers.

Third, it is important to sustain the interest and attention of students by providing facilities or channels for frequent and almost real-time discussions with the instructor and among themselves. The discussion forums in our MOOCs were seen to be a crucial ingredient in fostering such an environment. In addition, it is important to be mindful of the number of notifications sent out to students, as large amounts of emails from instructors or course administrators may deter students from reading them altogether.

Next, other more detailed recommendations for future rounds would be to explore the possibilities of making all modules available to students to provide more flexibility allowing some students to finish sooner, and also allowing students to the only register for one of the two courses offered as taking two courses might overburden students and increase noncompletion.

Finally, there needs to be more systematic research on what behavioral incentives can lead to an increase in completion rates, which must be incorporated even before a course is launched.

ADB will be partnering with FAO again and other agencies to rerun this course in the second quarter of 2020. 

Asian Development Bank. 2020. Massive Open Online Courses for Building Statistical Capacity in Computer-Assisted Surveys. ADB Briefs. No. 137. Manila.

Development Asia. Explainer: How Mobile Technology Enables Better SDGs Monitoring.