Printer-friendly VersionEmail to a Friend

A Skewed Picture

Are the Millennium Development Goals presenting an accurate picture of progress?

SEE-SAW STATISTICS Nepal has reduced the under-5 mortality rate by half to 74 per 1,000 live births over a period of 15 years. In the country’s Midwestern region, however, the rate is much higher than the national average at 122 per 1,000 live births.
Photo by AFP

Nepal’s progress toward achieving the Millennium Development Goal (MDG) of reducing mortality rates in children under 5 looks pretty impressive. The country reduced its rate by half between 1990 and 2005 to 74 per 1,000 live births.

But that is little comfort if you live in the country’s midwestern region, where the under-5 mortality rate is dramatically higher than the national average—122 per 1,000 live births.

With an impressive national average in this indicator but lagging numbers in one region, should Nepal still be considered on track to achieve this goal?

Two-thirds of the way to the global MDG deadline of 2015, that is the question many government officials and experts around the world are asking about their countries. The national indicators of MDG progress provide a snapshot of the situation in a country. They do not reveal vast disparities within countries, particularly in Asia and the Pacific, home to three out of four of the world’s most populous countries.

Critics specializing in monitoring and evaluation argue that the MDG progress indicators inherently present a skewed By Bronwyn Curran picture. They are aggregate indicators, portraying progress at the national and international levels. But in countries the size of India or the People’s Republic of China, wide variations may exist between provinces and subnational groups. Critics say this puts into question claims by governments of progress in certain areas, and indicates the need for MDG progress to be assessed at the regional or provincial level.

When asked if the MDG indicators are too aggregative, South Asia Regional Policy Adviser of the United Nations Children’s Fund (UNICEF), Gabriele Koehler, responds: “Absolutely…They give a false impression that countries are on track when for some groups or geographical areas they are not, so the picture can be very misleading.

“They need to present a subnational picture along ethnic, linguistic, geographic, and religious lines, and those groups that tend to be excluded.

“For example, India is on track for the education MDGs, but when you disaggregate the indicators by ethnicity or religion or parts of India, there are many segments of the population where they are not on track at all,” Ms. Koehler explains.

United Nations Under-Secretary-General for Communications and Public Information Kiyo Akasaka acknowledges that aggregates can “mask differences which may exist within a group,” citing an example in Brazil, where the 2006 national average for access to improved drinking water was 91%, while the average in rural areas is 58%.

“Some countries show more than 30 percentage points difference between rural and urban populations” in relation to the MDG indicator on access to improved drinking water, Mr. Akasaka says.

But it works both ways, he points out. “Groups or populations faring well may also have too little weight on the national average. Looking again at the MDG indicator on access to improved drinking water, in Niger, for example, the 2006 national value is about 40%, but in urban areas it is more than 90%.”

To counter the potential distortion arising from aggregated data, UNICEF has created “Devinfo” maps that illustrate the disparities within countries in their MDG progress.

“Through our Devinfo maps we can see, for example in India or Nepal, how MDGs are doing in different parts of those countries,” he says.

UNICEF’s education section has also developed a parity index that shows differences by gender for education, and rural–urban differences.

The United Nations Economic and Social Commission for Asia and the Pacific (UNESCAP) monitors MDG progress in Asia and the Pacific, in partnership with Asian Development Bank (ADB) and the United Nations Development Programme (UNDP).

UNESCAP’s Regional Adviser on Statistics Jan Smit rejects suggestions that the MDG indicators distort the real picture.

“National-level MDG indicators measure what they intend to measure: a country’s overall progress towards specific MDG goals and targets. In that sense, they do not present a skewed picture,” Mr. Smit says in an interview.

“They do, however, mask withincountry disparities, which indeed may be substantial in some countries and may exist along geographical lines, but also along other dimensions such as sex, ethnicity, religion, age, or urban–rural locations. In order for governments to be able to target vulnerable groups and those otherwise left behind in overall national progress towards MDG achievement, the availability of data on within-country disparities, and how they are evolving over time, is critical.”

However, a country’s ability to disaggregate indicators may be limited by technical factors. “For example, it might not be possible to estimate under-5 mortality at an acceptable level of statistical precision in provinces with small populations,” Mr. Smit reckons.

The ESCAP–ADB–UNDP Asia-Pacific MDG partnership regularly addresses intra-country disparities in its regional rogress tracking reports. The Millennium Development Goals: Progress in Asia and the Pacific 2007 devoted its second half to subgroup disparities. It gave as an example the strong international performance in the goal of reducing under-5 mortality by two-thirds.

The 2007 report found that under-5 mortality rates are at least 50% higher in rural than in urban areas in Asia and the Pacific. While the Philippines is on track to meet the goal of reducing child mortality, the rural–urban divide is widening: in 2003 in the largely urban National Capital Region, the under-5 mortality rate was 31 per 1,000 live births, while in one largely rural region in the center of the country, the rate was 68.

Within urban areas the difference is stark between rich and poor households in the Philippines. Under-5 mortality rates are 52 per 1,000 in the poorest urban households, compared to 18 per 1,000 in the richest urban households.

Mr. Akasaka says the UN’s Statistics Division and the Inter- Agency and Expert Group on MDG Indicators (IAEG), responsible for preparing data and analysis on MDG progress, have encouraged data disaggregation wherever relevant.

“The MDG framework indicates that ‘all indicators should be disaggregated by sex and urban–rural as far as possible,’” the Under-Secretary-General says.

Faiza Effendi, who heads the poverty reduction and gender unit at the UNDP in Pakistan, says the country does not adhere to MDG reporting guidelines because they encourage the provision of disaggregated data by region and gender.

“The issue is that many developing countries do not have data broken down by region. Even in Pakistan, it is only in the last 3 years that the government has offered some district-level information, but then it is only in a handful of sectors, and that information is not broken down by gender,” Ms. Effendi says.

“It’s important to bear in mind that despite reporting through aggregate data, the MDG indicators manage to highlight the development trends or results in a country, be it on female primary school enrollment, female mortality, or child mortality, etc.”

The official UN website for the MDG indicators (http://mdgs.un.org) presents some indicators for urban and rural populations, others by sex, to show subnational disparities.

“The UN’s Handbook on MDG Indicators is currently being updated by the IAEG and will include guidelines on relevant disaggregations for each indicator,” Mr. Akasaka says.

In addition, the UN’s Statistics Division has held two workshops in Asia and Africa on MDG monitoring. Representatives of national statistical offices were trained in the production of data for MDG subnational monitoring.

Ms. Effendi says UNDP is supporting the Government of Pakistan in undertaking poverty head-count calculations by province, in developing a poverty social impact analysis—which shows development results and gaps—and in supporting poverty monitoring at national and subnational levels.

Oxfam, one of the major NGOs working on poverty eradication and gender equity, believes MDG indicators provide important barometers.

“They are the most comprehensive global framework we have to tackle extreme poverty, and as such they are an important first step towards eradicating such poverty,” says Oxfam Australia policy director James Ensor.

“The question shouldn’t be about whether MDG indicators are perfect, but whether the world is prepared to put in the political will needed to achieve the goals.” •