Designing Online Courses for Statistical Capacity Building
Published: 08 June 2020
Tapping the potential of MOOCs in training statisticians on tablet-based data collection yields promising results.
Conventional approaches to statistical capacity building typically involve organizing in-house training workshops. While this is a tried and tested method for knowledge sharing, organizing workshops entail a great deal of logistics, time, and money.
Technology has revolutionized nearly all aspects of our lives, including knowledge sharing and learning. With the advent of massive open online courses (MOOCs) offered through web-based platforms such as Coursera, Udacity, and edX, learning has been democratized, thereby enabling both college students and professionals to take courses most tailored to their interests. MOOCs have the potential of reaching an unlimited audience, given that the only requirement to take an online course is access to a computer with an internet connection. Furthermore, the MOOC approach allows learners from all over the globe to access the training course material at their own convenience, alongside the ability to interact with fellow learners through online forums, instant messaging, online calls, and numerous other interactive features. This provides a flexible environment for full-time working individuals, like the national statistical office (NSO) staff, who might have a limited amount of time during workdays to take the course.
This article offers insights on what works best in designing and implementing MOOCs based on the Asian Development Bank’s (ADB) largest online training initiative to build statistical capacity.
Two Courses over a Period of 8 Weeks
To leverage the numerous benefits associated with MOOCs for statistics capacity building, ADB organized two MOOCs in partnership with the Food and Agricultural Organization of the United Nations (FAO). The courses were targeted at enhancing the capacity of NSOs on tablet-based survey data collection, also known as computer-assisted personal interviewing (CAPI), but they were also open to anyone in the world interested in learning about this method. CAPI offers a cost-effective solution for gathering field data and generating timely and accurate statistics.
The aim of the MOOCs, like in-class training, was to equip learners with a solid understanding of how to develop and implement CAPI surveys using two popular platforms: CSPro for Android and Survey Solutions. The two courses ran over an 8-week period from 28 January to 22 March 2019 and were organized into eight sequential learning modules that were released on a weekly basis.
Each module contained lectures, as well as exercises and quizzes, that formed the basis for assessing learner performance. By the end of the course, learners were expected to at the very least grasp the fundamentals to develop CAPI questionnaires of their own. An optional capstone assignment was also offered at the end of the course to tests learners’ skills to program a sample questionnaire. We incorporated analytics to track learner progress and completion rates.
Constraints to Online Learning
One of the criticisms of MOOCs is that course participation is almost entirely in the world’s most affluent countries. This is attributed to costs associated with getting official certification for having completed a course. Also, internet access can be problematic in some parts of the world.
Another criticism of MOOCs lies in its low completion rates—hovering around 3% to 6%. This is likely because course developers may not be creating the right incentives to sustain the interest of learners over time.
To address these concerns, we made the courses free to all participants and permitted downloading of videos and course transcripts. We also incentivized learners by offering an all-expense-paid trip to the top four- performers in the capstone assignment to participate in a regional workshop on CAPI that was organized by ADB in Bangkok, Thailand.
High Participation Rate
The course was marketed through numerous channels, including a crisp promotional video, brochures, and flyers disseminated on social media. We also reached out to other multilateral agencies to spread the word. These efforts led to a total enrollment of 655 learners, which is unusually high for a niche topic like statistical capacity building.
Nearly three-fourths of the enrollees came from Asia and the Pacific, while a fifth were from Africa. NSO staff accounted for more than a quarter of the enrollees, while other government staff accounted for nearly a fifth of the sample. A third of the 655 learners enrolled only in the CSPro course, while a fifth enrolled only in the Survey Solutions course, and nearly half of learners registered for both courses.
Figure 1: Learners' Country of Origin
In terms of completion rates, 34.7% completed the CSPro course, while 36.6% finished Survey Solutions; that is nearly 7 times the average completion rates for MOOCs! A 100% completion rate was observed for Module 1, but the biggest drop was observed in Module 2, with 63.1% and 60.4% of the learners completing the Survey Solutions and CSPro courses, respectively. This can be attributed to the slight increase in complexity by Module 2. The next largest drop was seen between Modules 2 and 3, with completion rates at nearly 50% for both courses. Learner engagement data also showed that for both courses, those who made it through at least Module 4 were most likely to finish the entire course.
Figure 2: Completion Rates Over Time
The four students who attained the highest overall scores for both the modules and the capstone assignment were sent to the workshop in Bangkok on 25 November 2019. Two winners were selected from each course: May Myint Bo from Myanmar and Christian Mina from the Philippines for CSPro and Roshini Abeyesekera from Sri Lanka and Samuel Nogbou from Ivory Coast for Survey Solutions. The winners presented their projects and provided valuable feedback from their respective online course experiences.
Lessons and Recommendations
Based on student feedback from a survey administered at the end of the course, the MOOC platform was well-received and judged to be an effective tool for training a new generation of statisticians on CAPI.
The experience also generated a few lessons and recommendations moving forward.
First, ADB’s experience showed that MOOCs can reach a wider audience at substantially lower per-capita costs than in-class statistics capacity building activities. Further, strategic partnerships with other agencies maximized the use of resources in developing MOOCs on topics of mutual interest and widened the network of contacts to reach more people. Correspondingly, the wider reach of the MOOCs also enabled it to cater to typically underserved groups, such as women as well as individuals from low to lower-middle-income economies.
Second, our course was offered only in English, but the opportunity is seen in expanding the course offering to other languages, which would open doors to higher enrollments from non-native English speakers.
Third, it is important to sustain the interest and attention of students by providing facilities or channels for frequent and almost real-time discussions with the instructor and among themselves. The discussion forums in our MOOCs were seen to be a crucial ingredient in fostering such an environment. In addition, it is important to be mindful of the number of notifications sent out to students, as large amounts of emails from instructors or course administrators may deter students from reading them altogether.
Next, other more detailed recommendations for future rounds would be to explore the possibilities of making all modules available to students to provide more flexibility allowing some students to finish sooner, and also allowing students to the only register for one of the two courses offered as taking two courses might overburden students and increase noncompletion.
Finally, there needs to be more systematic research on what behavioral incentives can lead to an increase in completion rates, which must be incorporated even before a course is launched.
ADB will be partnering with FAO again and other agencies to rerun this course in the second quarter of 2020.
Asian Development Bank. 2020. Massive Open Online Courses for Building Statistical Capacity in Computer-Assisted Surveys. ADB Briefs. No. 137. Manila.
Development Asia. Explainer: How Mobile Technology Enables Better SDGs Monitoring.
Leave your question or comment in the section below:
YOU MIGHT ALSO LIKE
The views expressed on this website are those of the authors and do not necessarily reflect the views and policies of the Asian Development Bank (ADB) or its Board of Governors or the governments they represent. ADB does not guarantee the accuracy of the data included in this publication and accepts no responsibility for any consequence of their use. By making any designation of or reference to a particular territory or geographic area, or by using the term “country” in this document, ADB does not intend to make any judgments as to the legal or other status of any territory or area.