Should bad data mean no action?

By Claire Melamed 09 August 2016

Dan, a community health worker, logs details of malnourished children to record their recovery. Photo by: KUAP Pandipieri / CC BY-NC-ND

After decades of indifference there is now an overwhelming consensus that much of the data with which governments, companies and civil society organizations make decisions are pretty flawed. A plethora of reports and blogs — not least the report of the U.N. Secretary General’s International Expert Advisory Group on the Data Revolution, which I authored — have argued that there is too little investment in data, that too many people and things are missing from current data and that the use of new technologies to improve the situation will be slow and is marked by inequalities.

As a result, data and statistics that are commonly used and cited by all and sundry are seen as flawed at best, and at worst, misleading.

So what to do? Much is happening: Governments are experimenting with new approaches and new capacity. A new global partnership has brought together the governments, international institutions, companies and civil society organizations with an interest in solving these problems. New investments, such as the Bill & Melinda Gates Foundation’s $80 million for gender data, are increasing the overall resources for the sector.

There’s a long way to go before we can say data is good enough, let alone good, but the political focus has paid off in greater action, and the onus is on all involved to ensure that the energies are expended in the right way and for the right outcomes.

But in the meantime? In a Devex commentary, Morten Jerven has argued that the data is so poor that attempts to monitor progress globally, for example on income poverty, are misleading, and that governments should not be encouraged to become more data-driven if the data is imperfect.

It is right to caution against fantasy data and wishful thinking. But to argue that no data is better than poor data, at global or at national level, seems to me excessive. I would prefer to argue a more optimistic approach.

First, there is a large gap between poor data and wholly useless data. Take, for example, maternal mortality — one of the issues on which data is agreed to be particularly poor. The numbers may be terrible and reflect not just the complexity of the issue but also decades of underinvestment in data on women. But even within the very large margins of error that this data involves, it’s clear that the trend is downward. The data tells us something — even if it’s not enough. Comparing trends, with appropriate deference to confidence intervals in different countries, can help to understand the value of different approaches to maternal health and inform debates on how much money is needed and what works.

Secondly, there is a gap between poor methods and wholly useless methods. There’s a consensus that there’s more to poverty than money and that it’s difficult to work out exactly how to capture that. Many people are trying. But that doesn’t mean that income poverty is not a useful way of thinking about poverty. Knowing how much money people have and whether that is changing over time is useful. Income poverty is correlated with other things such as health and education outcomes, so it’s an imperfect but helpful proxy for development more broadly. And, even without that, money does matter — just ask anyone who doesn’t have any. Giving people money, through cash transfers or other social protection schemes is one of the few development programming ideas that seems to really work, in many different contexts. So knowing who has what is a pretty essential starting point for interventions.

It may be a truism but it’s worth saying — money matters to poverty. So spending time, money and intellect on the measurement of income poverty is a good thing, not a pointless endeavour. And finding a way to compare trends and levels across countries is useful too. As with so many things in development, the tendency to let the perfect be the enemy of the good, can do a lot of damage.

The data that we have, even in its current state, can shed a light on problems and provide a guide on how to fix them. For political decision-making, politics will always matter most, and rightly so — people, and the governments they elect, set the priorities for action and the direction of travel.  That’s informed by things that can be counted and things that can’t.

But data is one of the tools that governments can then use to take people where they want to go. Saying that because you can’t count everything and because you can’t count well, you should ignore the value of counting anything at all, is taking data despair to too high a level. I think we can do better.

With potential to change the trajectory of crises, such as famines or the spread of diseases, the innovative use of data will drive a new era for global development. Throughout this monthlong Data Driven discussion, Devex and partners — the Agence Française de Développement, BroadReach, Chemonics and Johnson & Johnson — will explore how the data revolution is changing our approach to achieving development outcomes and reshaping the future of our industry. Help us drive the conversation forward by tagging #DataDriven and @devex.

About the author

Oped clairemelamed ed
Claire Melamed

Claire Melamed is a managing director at the Overseas Development Institute in London, U.K. where she leads work on poverty and inequality. She was previously head of policy at ActionAid UK. She has also worked for Christian Aid, the U.N. in Mozambique and taught at the University of London and the Open University. Claire's work focuses on measurement of poverty and inequality and on how to use the insights from measurement to improve policy and outcomes.

Join the Discussion