Achieving the 2030 sustainable development agenda requires development organizations to move beyond the “one-size-fits-all” model of development and the standardized approach typical of the industrial age. Instead, it should move toward interventions that are both bespoke and suitable for the complexity of the world we operate in and, at the same time, responsive and adaptable to a rapidly changing environment.
More reading:
► How to incorporate resilience thinking into policy making, development programming
► Should your NGO hire a data scientist?
► With aims to assign a dollar figure to data, NGOs have a new financial avenue
Traditional models of intervention are showing their limits, with persistent gaps separating carefully articulated policies from their realized outcomes. Similar to industrial age corporations, international development organizations have proven to be well suited to address specific problems through standardization — boiled down to best practices — but have largely failed to respond to the complexity that stems from the heterogeneity of problems, solutions, and contexts with which development concerns itself.
Meeting the challenges posed by the 2030 agenda requires development programs to be:
1. More differentiated
Programs must move closer to the ground by cultivating a real-time understanding of the context in which they operate at the community level. Doing so requires using both qualitative and quantitative methods to rigorously measure context and the outcomes achieved by programming. Once this is done, activities can be more effectively differentiated based on local demand and their achieved results leading to a less one-size-fits-all approach and one that is more likely to be owned by affected populations and leaves no one behind.
2. More adaptive
Programs must be able to learn and respond to ever-evolving situations and correct course implementation as needed. Research, monitoring, and evaluation approaches have gone a long way in understanding the efficacy — or lack thereof — of development programs and have contributed to a growing evidence base around key performance indicators. These approaches allow the development community to better measure whether outcomes are achieved. Their shortcoming, however, is that these approaches do not help us ensure an outcome is achieved.
“Data-driven adaptive management has the potential to transform the development industry.”
—Research, monitoring, and evaluation are de facto firewalled from program implementation rather than embedded in programs and directed toward achieving outcomes. Traditional monitoring and evaluation approaches present data after implementation is complete, when it is only useful for designing a new program instead of improving an existing one. With program durations spanning up to five years or more, losses in efficacy are greatly compounded. Development implementers rarely know where they stand relative to their objectives. Data is regularly used to measure what happened but rarely to make it happen.
Embedding data-driven adaptive management in development programs
What is data-driven adaptive management?
Data-driven adaptive management is the process of building real-time, high-resolution datasets and analytics that help optimize the achievement of an objective. It is an optimization process unique from M&E in that it seeks to go beyond measuring results and begin increasing the likelihood that they are achieved. As activities are refined and optimized, they become more demand-driven and differentiated at lower and lower units of analysis, e.g., communities or individuals.
Data-driven adaptive management processes, usually called applied operations science by the private sector, are responsible for the disruptive innovations of technologies ranging from streaming video to personalized medicine and digital marketing to ridesharing.
The lack of an efficient, accurate — high-quality and highly-structured — and affordable way of creating high-resolution big data sets specific to each program’s objectives is a key barrier preventing the adoption of data-driven adaptive management in international development. With the development of effective crowdsourcing technologies, the need to rely on enumerators and high margin-of-error sampling methodologies is removed, making the creation of real-time, granular data sets possible.
Earlier crowdsourcing examples utilizing big data involved the analysis of traffic on social media and produced highly unstructured data. This type of data was never generated with the intention of helping development programs and must be repurposed for such analysis. Further, social media data presents quality issues that don’t easily lend themselves to the more rigorous standards and the decision-making requirements of development M&E.
By embedding data-driven adaptive management into a program, and developing analytical models specifically tailored to its objectives, the data output is structured to meet the program’s requirements and facilitates learning and adaptation. This begins with the development of a hypothesis and a supporting analytical model for each key performance indicator at the outcome and impact level that can be measured quickly and at the lowest geographic level at which the program operates.
It is then critical for the organization to support management process to interpret and make sense of data iteratively during implementation, addressing the hypothesis at the heart of the program and formulating new, emerging ones. Importantly, it also contributes to the closer integration of monitoring, learning, and evaluating with program implementation, a necessary condition for an adaptive approach. When combined, the result is an ability to optimize activity implementation for increase impact.
How UNDP is piloting data-driven adaptive management
Data-driven adaptive management in action
Colombia’s cities are giving their Zika and Dengue vector control systems a data-driven, adaptive overhaul. All operations are now driven by a clear, outcome-based KPI: Maintaining the presence of the Aedes aegypti mosquito at less than 5 percent at a neighborhood level. Data to populate this model is gathered by a mix of formal vector control teams and ordinary citizens using smartphones so each neighborhood’s prevalence rate is monitored in real-time. Based on this data, inspection and remediation teams are deployed, with a focus on neighborhood, near the outbreak threshold. The result is a decrease in average prevalence in high-risk neighborhoods from 38 percent to 3.5 percent.
To adopt data-driven adaptive management, programs must also put in place regular management processes that embed learning to reflect on results and outcomes, and must leverage flexible and agile implementation methodologies. This approach draws on the structured experiential learning model and on early experiments, practitioners collect data use information in real time with direct feedback into the decision loops.
Collecting evidence in real time as well as analysis of lessons learned, allows for informed adjustments, including midcourse corrections, and may lead to program optimization, improved implementation, and improved outcomes. However, programs that endeavor to implement flexible and iterative approaches often suffer from lack of robust evidence and rely on a small number of directly employed staff to provide context analysis in the form of anecdotes to understand changing contexts.
To get the most complete picture of the situation on the ground, The United Nations Development Programme is partnering with advanced data providers such as Premise Data. The collaboration allows projects to advance contextual knowledge through the creation of an end-to-end data collection and analytics methodology to increase program impact. To test the approach, UNDP is launching pilots to integrate evidence coming from real-time data into existing projects to improve the effectiveness of the interventions.
Data-driven adaptive management has the potential to transform the development industry. In the future, we may see project plans become less detailed and be better able to accommodate changes in the environment, beneficiaries’ priorities, and our understanding of the problem and its solutions. Quarterly reports, evaluations, and success stories will remain important, but will stop acting as a substitute for increased impact today.
