In an apparent attempt to throw off the current economic gloom, U.K. Prime Minister David Cameron has asked government statisticians to try to come with a better measure for quality of life than gross domestic product. The tabloid press last month labeled his new indicator the “fun index,” but Cameron insists it’s “just a general picture of how life is improving.”
His intervention follows Nicholas Sarkozy’s appointment of Joseph Stiglitz to redefine economic progress, and the recent explosion of development-related indices published by think tanks, NGOs and others trying to measure concepts as nebulous as good governance, freedom or contentment.
Indices, especially composite ones, might be condensed measures of complex issues, or “mashups,” as Martin Ravallion, the World Bank’s head of research, has called them recently. He rightly warns against conceptual fuzziness, policy irrelevance and misusing weighting in indices. But as the person recognized as the architect of the World Bank’s “$1 a day” measure of poverty, Ravallion will also understand that to get your point across on international development, you often need to simplify.
To criticize indices for being crude tools overlooks their very point, which is to simplify complexity. In some ways the “$1 a day” measure helped establish a new definition of poverty. Often, advancing an issue or idea by redefining it is what an index is trying to do.
Sometimes indices are really about putting something on the map in first place. Take human development. The Human Development Index, the latest incarnation of which was released just a few weeks ago, is a seminal index. It has helped define Human Development, and has become a proxy for it. For good or ill, without the help of the HDI, Human Development may never have entered the mainstream development debate.
A fascinating extrapolation of the HDI in visual form has been crafted by Cesar A. Hidalgo and Alex Simoes of MIT’s Media Lab. This application helps show it is still useful to policymakers to understand the broad sweep of changes and comparisons in human development.
Indices are intrinsically political and the HDI is no exception. They don’t need to be perfect because they are part of an argument. That argument may well be controversial, like the Heritage Foundation’s Index of Economic Freedom, which penalizes countries, including developing countries, for “large” amounts (over 30 percent of GDP) of government spending or for having moderate labor rights. But if you believe that a significant, interventionist state inhibits freedom, then that’s your call, as long as you are upfront about it.
Ask any organization that has published an index and folks there will regale you with stories of ambassadors calling and countries changing policies. Indices have become powerful policy tools. Powerful they may be, but they can be used in troubling ways. Is it wise for the Millennium Challenge Corp., for instance, to use the Heritage Foundation’s trade-related indicator as well as other indicators (many from indices) to determine funding eligibility?
Indices can also be methodologically flawed. An index which chooses to compare countries where data is poor or non-existent with those that have comprehensive data collection can produce meaningless results. Equally irrelevant are those, such as Newsweek’s recent “Best Countries” index, that tend to overreach when trying to define a word like “best.”
Worst of all are unintended consequences. What outcomes does your index seek to produce? Are you measuring the right things? Can countries climb the ratings without actually improving the things you want? If you lock in bad practice, your index could be counterproductive.
In the end indices are just tools. Like many aspects of social science, and especially economics, they are pseudo-science. The key to a “good” index is the robustness of the argument used to put it together. As long as the argument is sound and the index based on robust data and reasonable methodology, indices can still ultimately add rather than subtract from the development debate.
But when you look at an index, be sure to look at the argument it is supporting, as well as the index itself.
