M&E 2.0: Using big data to kill white elephants

Development professionals should better harness 'big data' to improve project outcomes. Here, a Sudanese government employee gathers data on water levels at a camp for displaced persons in Darfur. Photo: UNAMID, (CC BY-NC-ND 2.0)

The transformative power of free markets and foreign direct investment in emerging markets is well-known and generally accepted. The socio-economic impact of international development interventions, however, is often questioned.

The practice of international development is an inherently complex and inexact science.  Indeed, by the time development professionals account for the potential impacts of cultural, social, economic, political and even religious factors on project design and likely outcomes, development looks more like an art than a science. 

Modern development projects, like those directed by DfID, AusAID and USAID, are therefore built with great care and thorough consultation with aid recipients.  Project designs and implementation frameworks leverage best practices, and weigh the likely impacts of the intervention on everything from minority groups to migratory birds. 

Yet modern international development design and implementation lacks a correspondingly modern monitoring and evaluation tool, one that would rapidly, effectively and inexpensively provide feedback on key performance indicators and outcomes. 

Such a tool would ideally provide development practitioners with early headlights on anticipated project outcomes, giving them an opportunity to course correct, to enhance positive outcomes and mitigate the negatives.

While M&E of international development projects is often discussed, in practice it is rarely undertaken.

Several factors help explain the absence of comprehensive M&E of development work: M&E is an expensive, labor-intensive and time-consuming endeavor, conducted on a post-hoc basis, which yields results too late for course correction.  Moreover, it often reveals incomplete or irrelevant data.

A new and better approach for measuring the impact and outcomes of international development projects – let’s call it “smarter impact” –  would present stakeholders with a tool not only to diagnose problems but also to predict and avoid problems before they occur.

Sound like science fiction? It’s not. 

When it comes to data analysis, much has changed in the last decade, even the last few years.  Crowd sourcing, social media, mobile technologies are all low-cost and widely available methods of collecting, analyzing and disseminating information  

The emergence of “big data” as a science; and the corresponding application of analytics to massive amounts of unstructured digital information, are all trends that open up new frontiers for the analysis and use of development data. 

To give just one example of how these data can transform real-world economic activities, predictive analytics can be used to provide granular weather forecasts that enable farmers in a particular location to optimize their planting, harvesting and crop protection decisions. 

Some from the international development community have argued that in places where the paper ledger still reigns supreme – including in many developing countries – electronic data remain too sparse and inconsistent to be useful. 

However, this too is changing as businesses modernize, and more governments take advantage of “e-gov” solutions, like e-health and e-social services to help cut corruption and improve the delivery of citizen services.   e-government systems are put in place.  And certainly with the ubiquity of mobile devices, cellular communications, and innovative “apps”, new alternatives to traditional uses of data abound. 

Just as complex fields like medicine, financial services, transportation and logistics have embraced big data analytics, the traditionally technology-resistant international development industry should explore the possibilities for improved impact evaluation that could be unleashed through new analytical tools and techniques.  

The status quo of post-hoc, or even more common, of no impact evaluation for international development interventions, must change. 

In this era of breathtaking technical advance, fiscal austerity in the global north and persistent poverty in the global south, every development dollar, euro, and rupee must count. 

White elephants, roads to nowhere and well-intended development flops must become a thing of the past.  New tools and technologies must be harnessed, embraced and brought to bear on the practice of international development for the benefit of all.

Explore related content:

Join Devex, the largest online community for international development, to network with peers, discover talent and forge new partnerships - it’s free! Then sign up for the Devex Impact newsletter to receive cutting-edge news and analysis every month on the intersection of business and development.

About the author

  • Tim Docking

    Tim Docking leads IBM's Emerging Markets Funding group, a center of competence on international development finance. Prior to joining IBM in 2008, he was Senior Advisor to the CEO of the Millennium Challenge Corporation.