How Data Systems Can Lead to Better Program Monitoring and Evaluation

Staff can access accurate, consistent, and timely reports through a data system. Photo credit: Teach for the Philippines.

Share on:           

Published:

A centralized platform that promotes data sharing and visibility can help improve an organization’s efficiency and impact.

Introduction

Teach for the Philippines is a non-profit organization that works to provide all Filipino children with access to inclusive, relevant, and excellent education. Founded in 2012, it focuses its efforts on improving teacher quality and addressing education challenges at the system level. Through its three core programs—Fellowship, Public School Teacher Pathways, and Ambassadorship Programs, the organization creates a pipeline of teachers who can significantly improve student learning outcomes, catalysts who can spark change in the community, and leaders who can set reforms in motion to transform the public schools. It partnered with the Department of Education to enlist new and current teachers to join these programs.

In 2016, the Data and Impact Assessment team was created to enhance the organization’s operations and eventual impact and to provide empirical proof of its programs’ success. This cross-functional team analyzes the monitoring and evaluation frameworks and helps create structure and consistency in data management. The team has also managed multiple data system projects to improve data collection, validation, and sharing within the organization using Google Workspace and Sheets. However, as Teach for the Philippines’ programs and operations evolved, including changes made in response to the COVID-19 pandemic, it had to revisit existing processes and data system technologies that support its research and evaluation agenda to ensure that they are responsive to the changing context.

This article shares insights from the Asian Development Bank’s Knowledge Immersion Program (KIP), which provided Teach for the Philippines support and guidance in establishing a data system to improve project monitoring and evaluation. Financed by the Republic of Korea e-Asia and Knowledge Partnership Fund, the program builds the capacity of ADB’s developing member countries by strengthening their knowledge and skills on information and communications technology (ICT). 

Analysis

Using data to improve performance

As a young organization, Teach for the Philippines seeks to continuously improve its programs using quantitative and qualitative data. For example, during the onset of the pandemic and school lockdowns, it had to navigate the changes that occurred and pivoted the delivery of its programs. It conducted a series of surveys that targeted education stakeholders, such as households, teachers, and education leaders. Findings and insights from the data collected helped inform the design of its literacy program to adapt it to the distance learning setup.

Alongside the change in program implementation, the method for data collection was revised as well and different online tools were used. Monitoring and evaluation tools, which run using Google sheets and App script, help teachers track the progress of their students and measure student outcomes by the end of the program. Apart from collecting student data, the organization annually conducts satisfaction surveys to ask students, their parents, and other members of the community about their perception and feedback on teachers and the work that they do.

Shifting from trackers to data systems

Teach for the Philippines has previously relied on trackers for data management. These are simple tools or software, usually Microsoft Excel or Google Sheets, used for monitoring and evaluation. For small organizations, they are adequate data management tools because they can be easily acquired, learned, and shared in a small office environment. They can also be changed based on the requirements of each program cycle.

However, the process of generating reports from trackers can be very complex. The data analyst usually combs through multiple software and tools, applies multiple rules only learned from experience, and manually creates reports for every month, quarter, and year. Other than the data analyst, no other staff member could create such reports due to the high possibility of inaccuracy and inconsistency.

In contrast, data systems can make the process of generating accurate, timely, and consistent reports much easier. While trackers focus on data collection, data systems focus on data analysis. Data systems usually consist of an intranet, a data warehouse (or information repository), and a web-based reporting platform that displays available data through a dashboard. The intranet and the data warehouse classify, standardize, validate, and automatically pre-process data that will be used for analysis. As this centralized web-based platform enables staff to access and share data across the organization, it can lead to more efficient and effective monitoring and evaluation of programs.

Over the last few years, the Data and Impact Assessment team has been leading Teach for the Philippines’ transition from trackers to a data system.

Building a data system requires significant effort from business stakeholders. They need to explain their business processes and data to the project team and provide timely feedback during all project phases. However, once it is built, the data system can provide accurate, consistent, and timely reports to all staff with minimal manual effort. As data could be easily accessible and analyzed, the efficiency and effectiveness of programs and the organization could be better monitored and evaluated. It can also show the value of individual staff members’ contribution and provide more insight to the programs by allowing complex data linking.

An organization should consider building a data system if

  • there are frequent requests for trackers from other teams and it takes some time to process these requests;
  • the data of individual teams is not known to the other teams;
  • the same data is collected and managed by individual teams;
  • too much time is spent on cleansing data for each report;
  • it is difficult to link trackers with different data granularity; and
  • very few staff can generate consistent reports.
Implications

All organizations, especially nonprofits, have limited resources, such as funding and human resources. Thus, they focus mainly on ensuring operational excellence and delivering impact rather than improving information technology (IT) systems. However, technology is necessary to bring innovations quickly. Data systems provide a lot of control and automation, which can only be achieved by technology. All organizations need an IT strategy and vision and give priority to innovating themselves.

Recommendations

In the short term, two activities will increase data sharing and visibility within organizations.

The first one is to create a data dictionary that shows the names and data types used by the teams. Data can be classified based on the following: i) reference data that all teams can use, ii) program-specific data that can be frequently changed but useful to other teams, and iii) sensitive data that cannot be shared outside of the team. The data dictionary should provide the metadata (or basic information on the data). The data dictionary will lead to identifying redundant data, linking data, and implementing consistent, efficient, and integrated data management.

The second activity is to document a few but important business processes. This is essential when aiming to reduce dependence on individuals and seeking to rely more on processes within the organization. Well-defined business processes are also integral to data collection and standardization. They will bring transparency and openness, which will make the organization more efficient, lean, and agile.

In the long term, it is important to have an IT strategy and funding. As the organizational effort should be directed toward learning the problem and finding solutions, IT can support the innovation the organization is trying to achieve. It would not be difficult to get financial support for IT projects that are essential to an organization’s operations. Hiring an IT expert can be considered as IT support for the programs and operations would require relevant experience and knowledge within the organization. Any IT project will burden the organization with additional work and resources, but a new IT system can improve efficiency and effectiveness.

Jae Hun Lee
Consultant (Knowledge Expert), Asian Development Bank

Jae Hun Lee is an ADB consultant to Teach For the Philippines under the Knowledge Immersion Program, financed by the Republic of Korea e-Asia and Knowledge Partnership Fund and administered by the Asian Development Bank. Through virtual deployment, he provides ICT support despite COVID-19 restrictions.

Asian Development Bank (ADB)

The Asian Development Bank is committed to achieving a prosperous, inclusive, resilient, and sustainable Asia and the Pacific, while sustaining its efforts to eradicate extreme poverty. Established in 1966, it is owned by 68 members—49 from the region. Its main instruments for helping its developing member countries are policy dialogue, loans, equity investments, guarantees, grants, and technical assistance.

Follow Asian Development Bank (ADB) on
Leave your question or comment in the section below:
Disclaimer

The views expressed on this website are those of the authors and do not necessarily reflect the views and policies of the Asian Development Bank (ADB) or its Board of Governors or the governments they represent. ADB does not guarantee the accuracy of the data included in this publication and accepts no responsibility for any consequence of their use. By making any designation of or reference to a particular territory or geographic area, or by using the term “country” in this document, ADB does not intend to make any judgments as to the legal or other status of any territory or area.