IFC's new way to measure development impact: 'An art as much as a science'

WASHINGTON — As part of its new mandate to put development impact at the heart of its investment strategy, the International Finance Corporation is introducing a new rating system designed to enable more informed judgements about the types of projects the IFC supports.

Speaking during a side event as part of the Civil Society Policy Forum at the IMF and World Bank Annual Meetings in Washington, D.C., Hans Peter Lankes, the IFC’s new vice president for economics and private sector development, unveiled a complex and sophisticated new approach to measuring and tracking development impact at the institution.

The Anticipated Impact Measurement and Monitoring (AIMM) system offers an “end-to-end” framework to help IFC managers and its board make decisions about which projects to approve based on a range of quantitative and qualitative information and estimates about the project’s likely impacts on poverty alleviation and market creation.

“We want to have the ability to select projects on the basis of the development impact we believe they have, and we want to have incentives to ensure that the projects presented for decision-making are projects with the strongest possible development impact,” Lankes said.

The new system, launched as a pilot in July, adds multiple layers of analysis, tracking, and measurement to the IFC’s current impact measurement system — Development Outcome Tracking System (DOTS) — which only looks at the direct impacts of projects and only three years after they start. AIMM also represents a change in thinking for an institution that some have previously accused of being more interested in profit creation than poverty alleviation.

“Development impact is multidimensional and qualitative and what we need is a well-structured argument and as many elements of judgement as we can have in order to come to these judgements,” Lankes said, “It is an art as much as a science.”

Civil society actors who spoke described AIMM as a step in the right direction, but also criticized the lack of civil society engagement. Groups also raised concerns about how AIMM would be applied to IFC’s investments in financial intermediaries.

“I think the fact that the less quantifiable qualitative elements around development impact are being considered is certainly a step inthe right direction,” said Luiz Vieira, coordinator at the Bretton Woods Project, an NGO that monitors and produces research on the World Bank and the International Monetary Fund. He welcomed the IFC’s recognition that development is a “fraught, complex process that requires more nuanced thinking.”

But he told Devex “it’s a pity this process has been developed with no outreach to civil society,” adding that local and international NGOs have the needed “deep understanding of community dynamics,” and of who is likely to benefit and who could be harmed by development projects.  

A direct feedback loop from M&E into project design

In addition to maximizing development impact, Lankes said that the AIMM system is designed to bring a number of additional benefits, including creating “direct feedback loops” between project evaluation findings and the way projects are designed and chosen. Under AIMM, lessons learned and information gathered through “downstream” project monitoring and evaluation will be fed back into the “decision-making process upstream in a more systematic way perhaps than has been the case,” Lankes said.  

Furthermore, the system will involve monitoring and evaluation staff in the decision-making process for projects before they get approved, thus creating a “very direct feedback loop between project evaluation into project design and choices,” he added.

AIMM will also create a “clear line of sight” from projects to the World Bank’s twin goals of ending extreme poverty and boosting shared prosperity, as well as to the Sustainable Development Goals, the IFC official explained. He said that under the current system, the correspondence between the goal of projects and the SDGs is not always clear.

Implementing IFC’s creating markets agenda

The IFC’s new assessment system can also help implement and track the IFC’s new strategy around creating markets and enabling the entry of the private sector into fragile countries, Lankes said. Instead of just focusing on a project’s direct impact on employment, for example, AIMM will seek to measure and track whether a project contributes to market creation, he said.

Lankes also said the institution hopes that AIMM will provide a platform for comparing potential projects in a way that is not currently possible. For example, while one project may have a high development impact but pose financial risks, this could be balanced by an investment with high financial returns but low development impact. “We have no way today to have that conversation with our board,” Lankes said.

New tools, assessments, and indicators

Implementing AIMM requires the development of new indicators and measurement models beyond those used under DOTS. In addition to the market creation indicators, IFC staff are also working to develop “probability assessments” to estimate the likelihood that a project will go as predicted “in order to correct for what might be an optimistic bias otherwise,” Lankes said.

The IFC team working on AIMM hopes to have a fully fledged system in place by July 2018. The existing DOTS system will eventually be integrated into the new system once the AIMM portfolio reaches a critical mass.

Devex reporters Michael Igoe and Sophie Edwards will be on the ground at the World Bank annual meetings. Sign up for our daily morning briefings and follow us on Facebook and Twitter for everything you need to know from the annual meetings.