Data revolution: It's not just about technology

The international development community has a data problem. The vast majority of information on progress toward Millennium Development Goals is either missing or based on estimates. Botswana is often cited as an example of this, as its poverty figures are derived from data collected in 1993. In other instances, official data differ largely from surveys conducted independently. In some cases, the most basic kind of information is missing. Only a quarter of South Asian countries, less than half of Latin American and Caribbean countries, and 6 percent of sub-Saharan African countries have complete civil registration systems. Registration of births and deaths has stalled globally. Afghanistan’s last national population census was conducted in 1979; the Democratic Republic of the Congo’s was in 1984. Data on gross domestic product, which influence everything from international aid to foreign investment, can be unreliable. This year, Nigeria’s economy was revealed to be 90 percent larger than previously thought after its GDP was “rebased,” meaning a new base year was used to reflect structural changes in the economy. In Kenya, a similar exercise led to a 25 percent growth in GDP. In fact, the majority of sub-Saharan African countries are measuring GDP using outdated base years, and much of the indicators that are part of their calculations are estimates at best. Because it hosts some of the world’s poorest countries, sub-Saharan Africa is especially affected by a lack of good data — what Shantayanan Devarajan, chief economist of the World Bank’s Africa region, has dubbed “Africa’s statistical tragedy.” Focus more on impact not innovation In its report titled “A World that Counts,” the U.N. Data Revolution Group called for a push in using innovation to generate improved data. Indeed, a number of exciting projects that have been led in the past few years point to the potential of using big data for development, from using geographic information systems mapping to calculate population growth, to using light emissions from space as an indicator of economic growth. A study looking at mobile phone data to track and predict malaria epidemics in the Ivory Coast yielded better results than epidemiological models in use. And a volunteer group called the Humanitarian OpenStreetMap Team produced a map for Monrovia from satellite images to help humanitarian organizations fight Ebola in the Liberian capital. Yet experts warn against putting too much emphasis on innovation. “Many people talk about the data revolution as a technology revolution,” said Amanda Glassman, director of global health policy and senior fellow at the Center for Global Development. “I think that the revolution is more around getting basic data produced.” Glassman became interested in looking at the impact of poor administrative statistics after noticing that much of the improvements in data collection in the health sector, such as randomized control trials, couldn’t be used at the policy level. In a study she conducted with her colleague Justin Sandefur about immunization and school enrollment rates in certain sub-Saharan African countries, significant discrepancies were found between official administrative data and household surveys conducted by the Demographic and Health Surveys Program, a project funded mainly by the U.S. Agency for International Development. They concluded that by tying aid to performance without ever checking that the numbers were accurate, donors created a perverse incentive for states to tweak the data. The cost of collecting and analyzing data Glassman said donors need to rethink the way aid goes to supporting the production of official statistics, beginning with the amount of aid that’s disbursed toward that goal — $500 million a year is not enough. “There certainly is a funding shortfall, and the funding that does come in is volatile and very much tied to donor priorities,” she told Devex. “We also see some perverse incentives in the way that donors fund things. They fund a lot of household surveys, they won’t pay for salaries but they will pay for field work, with the result that sometimes you see people just doing as much field work as they possibly can, and not doing the production of the basic statistics.” During the research for his book “Poor Numbers,” Morten Jerven, an associate professor at Simon Fraser University’s School for International Studies, visited the national statistics offices of eight sub-Saharan African countries, and conducted phone surveys with 30 others. He found underfunded, understaffed teams — when he visited the national accounts department of Zambia’s Central Statistical Office in 2007, only three people were working there; by the time his book was published in 2013, all of them were gone. While the temptation is there for donors to fund their own surveys or bypass the production of national statistics altogether, Jerven said development is undeniably tied to the ability of governments to produce good data and, ultimately, to assert their sovereignty. “What we’re talking about is whether states have the capability of knowing things about themselves, and moreover the ability to spread that information as the correct one,” Jerven told Devex. “If you own the data, you own the policy.” Economic data, beginning with reliable GDP numbers, have been suffering from the focus placed on social data, led by the need to measure the success of aid programs as well as progress toward the MDGs, and the problem might worsen as we move toward sustainable development goals next year. Jerven calculated that measuring every single MDG indicator between 1990 and 2015 would have cost $27 billion. With 17 new goals and 169 targets having been proposed by the U.N. General Assembly’s Open Working Group, the cost of reporting on the SDGs would amount to $254 billion. And yet the question of whether SDGs will actually be measurable, and at what cost, is hardly discussed. Quality vs. quantity In a report published earlier this year, the Data for African Development Working group, which Glassman co-chairs, called for donors to change their funding mechanisms in support of national statistical systems, including working toward reducing the dependence of national statistical offices on donor funding, increasing their autonomy toward governments and channeling donor money through governments rather than earmarked funding. The report also called for civil society groups, think tanks and nongovernmental organizations to monitor the quality of the data produced rather than focus on the quantity. Development organizations play a role in pressuring large institutions to stop disseminating inaccurate numbers as well. The African Development Bank, for instance, is being increasingly criticized for its estimate of the African middle class, which includes households living with an unrealistic amount of $2 to $20 a day. “There’s a role for international organizations and for NGOs [that] are working in the field and collecting independent data to create some reputational costs for reporting inaccurate data,” Glassman said. “If we know that the way agricultural productivity is reported by the Food and Agriculture Organization [is] based on increases in yields, and we know that’s really different from what surveys are producing, then it’s up to the community in agriculture and in food to put pressure on FAO to change the way they estimate the numbers, or at least the way that they report the whole set of numbers.” Check out more insights and analysis provided to hundreds of Executive Members worldwide, and subscribe to the Development Insider to receive the latest news, trends and policies that influence your organization.