Smart Ways to Collect and Share Data for Monitoring SDG Progress

Assessing progress in reducing inequality and promoting inclusion requires quality, timely, and specific data. Photo credit: ADB.

Share on:           


New technologies, including APIs and SDMX, are revolutionizing data collection and sharing, reducing the data gap in Asia and the Pacific.


Developing countries in Asia and the Pacific were already struggling to achieve the United Nations Sustainable Development Goals (UN SDGs). Then the pandemic triggered a sweeping human and economic crisis in 2020 that has resulted in many millions falling back into poverty and rising inequality. The SDG target date of 2030 is looming, so making up the ground lost to the pandemic and getting back on track are more crucial than ever.

We need as many contemporary tools as possible to help meet the targets, including smarter data collection. Open data helps ensure that plans to achieve the SDGs are evidence-based and that their outcomes are measurable. And we need common and comparable indicators across countries too.

For example, assessing progress in reaching  SDG 10 on reducing inequality and promoting inclusion requires quality, timely, and specific data, including indicators such as per capita household income growth and the income growth rate of households among the bottom 40% of the population. To be able to compare income inequality in two or more countries, data need to be compiled in a standardized way, based on a sound methodology and appropriate statistical procedures.

But there is a yawning data gap. This gap impacts most severely on the poorest and most marginalized—the very people we need to focus on to achieve zero extreme poverty and “leave no one behind.” From a policy perspective, the lack of quality data is problematic because it is hard to design programs that are meant to appropriately target vulnerable segments of society.

What APIs Can Do

One technology that is revolutionizing data collection and sharing is Application Programming Interfaces (APIs). An API allows machine-to-machine communication of data and provides ready access to data. APIs are everywhere, and we are all using them whenever we are online. Sharing text, audio, and video on social media or making payments over the web are two examples of their use.

The Asian Development Bank’s (ADB’s) Basic Statistics series and its Key Indicators Database source many indicators, including SDG indicators, using APIs from sources such as the UN’s SDG Global Database and many others. These APIs are a game changer: they make the process significantly more efficient by facilitating real-time, database-to-database information transfer, minimizing the need for human intervention and error. This goes a long way to ensuring data integrity.

What a contrast to the old days! Before APIs, collecting data was more akin to using the postal service: letters of request had to be sent to agencies or government ministries, followed by a wait for a reply and then often by manual uploading of statistics to the requestor’s database. In many cases, these old methods of collection and dissemination meant databases were out-of-date before they were even published.

SDMX: A Platform for Sharing Data

Another important tool is the ISO’s Statistical Data and Metadata Exchange (SDMX), a preferred standard for data exchange sponsored by the UN, World Bank, and others. SDMX has been successfully used to share SDG indicators. In addition to SDMX boosting standardization and automation in data collection, its universality is important too. SDMX is an evolving global initiative, with well-established governance and an international community that promotes shared tools and experiences.

The technology is effectively a box of different tools that are mainly free and open-source. Users can create, upload, query, or retrieve SDMX data through a graphical user interface or through an API.​ This flexibility leads to greater participation from important (but traditionally hard-to-reach) data suppliers, like community organizations, vulnerable communities, and ethnic minorities.

Datasets published in SDMX format are easy to share and consume by the users. Moreover, the data validation can be done automatically. Economies in the region that have already started to share their dataset in SDMX format include Fiji and Samoa.

SDMX-based APIs can extract statistical data accurately and quickly, and then share them on a number of platforms. The advantages of deploying them for better monitoring of SDG progress are many: faster access to data and metadata; globally standardized data; a huge reduction in data errors through automation; and lower IT costs due to the use of open-source software.

ADB’s Key Indicators Database uses SDMX and API technology to provide data for each of ADB's 49 member economies as well as selected indicators for the SDGs. These technologies are also instrumental in the process of collecting, analyzing, and sharing disaggregated data, splitting information into smaller units to highlight underlying trends and patterns.

Of the 84 indicators in ADB’s Basic Statistics 2022, 48 SDG indicators have been collected using APIs and include a degree of useful disaggregation, often by sex or region. Wider use of data-specific APIs and universal adoption of SDMX across Asia and the Pacific would go a long way to bridging the region’s data gap by ensuring the efficient and transparent transmission of country data, disaggregated data, and metadata for SDG indicators.

Sean Crowley
Freelance Writer

Sean Crowley is a freelance writer. He worked for Asian Development Bank’s Department of Communications in media relations, multimedia, and web for many years. Before joining ADB, he worked in communications for the UN and UNU-WIDER and as a journalist for the BBC and the South African Broadcasting Corporation. He has a master’s degree in Communications from the University of Tampere in Finland and a first degree from the University of Warwick in the UK.

Stefan Schipper
Senior Statistician, Economic Research and Development Impact Department, Asian Development Bank

Stefan Schipper’s work includes building statistical capacity in ADB’s developing member economies and using new digital technologies to improve databases. He also creates knowledge products, including statistical flagship publications. Prior to joining ADB, he was a statistical officer at the EU’s Eurostat in Luxembourg. He was a senior economist and statistician at the German Central Bank (Deutsche Bundesbank). He holds a doctorate and a master’s degree in Business Administration from the European University Viadrina, Germany.

Pamela Lapitan
Associate Statistics Officer, Economic Research and Development Impact Department, Asian Development Bank

Pamela Lapitan has a degree in Statistics from the University of the Philippines Los Baños (UPLB). Prior to joining ADB, she was officer-in-charge and statistical coordination officer of Regional Division IV of the National Statistical Coordination Board. She was also an instructor at the Rural Highschool and the Institute of Statistics of UPLB and worked for Bayer Philippines. Her interests are statistical analysis, planning and conducting surveys and experimental designs, geographic information system, poverty statistics, national accounts, and small area estimation.

Asian Development Bank (ADB)

The Asian Development Bank is committed to achieving a prosperous, inclusive, resilient, and sustainable Asia and the Pacific, while sustaining its efforts to eradicate extreme poverty. Established in 1966, it is owned by 68 members—49 from the region. Its main instruments for helping its developing member countries are policy dialogue, loans, equity investments, guarantees, grants, and technical assistance.

Follow Asian Development Bank (ADB) on
Leave your question or comment in the section below:

The views expressed on this website are those of the authors and do not necessarily reflect the views and policies of the Asian Development Bank (ADB) or its Board of Governors or the governments they represent. ADB does not guarantee the accuracy of the data included in this publication and accepts no responsibility for any consequence of their use. By making any designation of or reference to a particular territory or geographic area, or by using the term “country” in this document, ADB does not intend to make any judgments as to the legal or other status of any territory or area.