Opinion: Investing in ‘deep dives’ to measure development impact

There is a lot of faith in the private sector’s capacity and willingness to generate positive development impact in low- and middle-income countries. A new class of investors pursues such impact intentionally — alongside financial returns — while most development finance institutions explicitly use aid resources to nudge private investors toward producing public goods and generating societal impact.

Does such reinvented capitalism hold the promise of delivering development, the Sustainable Development Goals, and effective action to address climate change, loss of biodiversity, and more generally, global public good challenges?

Measuring development impact: An imperative

Measuring the actual development impact of private sector investments and activities is key to the answer. This seems to be well understood by all stakeholders, including private investors. Impact measurement has received increasing attention from private investors, notably focusing on the nature of reporting, the choice of indicators, and the need to harmonize measurement practices and approaches. Impact intention and impact measurement have also become part of an integrated impact management approach.

However, this still chaotic effervescence falls short of what is needed to identify impact, understand impact generation, and make impact investors more accountable for it.

The challenge is serious and multifarious. Many impact assessments mainly focus on inputs used in a given activity and outputs that it produces. However, outputs misrepresent the actual impact on society. Furthermore, understanding impact generation also implies assessing causality, i.e., whether or not the activity under study can be considered as a cause of the observed impact. Proving causality is a tough challenge because so many other potential effects need to be ruled out and the “counterfactual” — what would have happened without the activity — needs to be discussed.

This requires data and careful analyses based on economic and econometric modeling to take out various biases and tell consistent stories. Such studies are expensive, often several hundred thousand dollars, and require time as well as technical proficiency. It is unlikely that private investors — acting under business and profitability constraints — even when caring about development impacts, will spontaneously invest the needed resources. Finding cost-effective ways to mobilize evaluative research is a paramount objective.

Meeting that challenge has been the main objective of the European Investment Bank and Global Development Network program on applied development finance. The inspiration for the program was born out of the need to gain a better understanding of the development impact of a new set of EIB projects — which aimed to support private sector impact investors in Africa — and from the realization that none of the available evaluative products were adequate for that purpose. Evaluations are usually designed to investigate all relevant dimensions of a given project or program and do not focus on impact generation. Scientific “impact evaluations” are too heavy to be generalized and to provide timely operational insights.

‘Deep dives’ or how to mobilize academic research in a private sector context

A lighter product based on serious and contextualized evaluative research was needed and conceived on the following principles: it would be developed by promising LMIC researchers — preferably based in the countries of operation — building on their knowledge of the local context. This would send a powerful signal of salience, credibility, and legitimacy of the impact studies and contribute to the emergence of a culture of evaluative research, notably applied to private sector investments. The researchers would be mentored to conduct the studies by some of the best experts in the field. The program would thus build evaluative research capacity and bring its own human development benefits.

EIB and GDN’s resulting evaluative research papers — or “deep dives” — use research-based approaches to investigate the nature of the societal impact of private investment and related causality in a particular context. In the three-year pilot phase — taking place between 2018 and 2021 — 30 talented researchers from Africa and the Caribbean produced 16 deep dives, using projects funded by EIB under a special impact investing envelope. They covered projects in eight African and two Caribbean countries, ranging from microfinance to telecommunications, health and education, trade, and energy.

One of the studies estimated the willingness to pay for utility bills using mobile money. Another investigated whether a microfinance program had reached the poorest women and impacted women’s economic empowerment. One study looked at how free internet access provision to schools affected student outcomes.  

The deep dives had to productively reconcile the logic and demands of academic research as well as the professional objectives of young researchers shaping their careers: How to formulate research questions that would cater to private companies’ interests? How to conduct serious research and provide results in a timely way from their perspective? How to write research papers in a language that is clear, precise, and accessible? How to choose appropriate methods and investigate causality despite a general shortage of data? And, of course, how to formulate results in ways that satisfy both academic standards and private sector expectations?

In addressing these questions, maintaining academic quality was paramount and seen as a condition of credibility.

Connecting better research with action

The deep dives provided a much-needed proof of concept — it is possible to carry out research-based impact assessments that meet multiple stakeholders’ interests and provide useful information about the nature of development impacts. This can be done at a reasonable cost, with the additional benefit of building local evaluative research capacity in the process.  

This deep dive approach has great scaling-up potential. It also provides an innovative way to promote the connection between research and action. Research questions have to be agreed upon with non-researcher private partners, and to maintain their necessary involvement, they need to be involved and kept informed all along.

This interaction is essential to the deep dive process and is not limited to an ex-post communication of results, however important. This continuous interaction model provides a promising way to extend the culture of evaluative research to the private sector and rethink the organization of the evaluation function in development finance institutions.

Learn more about the EIB/GDN program and initiative.

This coverage exploring innovative finance solutions and how they enable a more sustainable future, is presented by the European Investment Bank.