How the IEG helps influence the World Bank's impact

The Independent Evaluation Group is tasked not only with assessing the World Bank Group’s effectiveness but identifying lessons that will shape its future strategy.

Devex spoke with IEG Director General Caroline Heider about the challenges of measuring IEG’s impact, findings from its latest evaluation into the World Bank Group’s work in situations of fragility and conflict, and the importance of preparing for a crisis.

“A key finding from IEG’s recent evaluation of the World Bank’s engagement in situations of fragility, conflict and violence is that the causes of conflict, the shape of the political economy and the identity of key stakeholders are often either not understood, are understood at a very abstract level, or are understood but not translated into a strategy or operational program,” said Heider.

The World Bank and its clients also need to build institutions and capacities in the good times, in preparation for the bad times, she added. “Once a crisis hits, trying to develop a social safety net is almost impossible because everybody is busy responding to the immediate disaster.”

Here are some more highlights from our conversation:

How does IEG’s work feed into the broader mission of the World Bank Group?

Evaluation is an instrument to help the World Bank Group be more successful, to have a greater development impact and produce better results for client countries and their people. Evaluation is not a means to control or punish, but a way to find opportunities to learn, overcome problems and fix systemic issues to produce better results. It’s about accountability and learning to avoid avoidable mistakes.

We’ve adopted two main objectives for IEG. One is to assess what the World Bank’s contributions are to achieving the twin goals of eradicating poverty and boosting shared prosperity.

Here, we point to areas where the Bank Group can push boundaries further and where it’s on track and should replicate efforts.

Our second core objective is to look at the World Bank Group’s internal workings and contribute to development outcomes by improving them. For example, IEG has looked into the World Bank reforms of 2013 to provide early feedback on how the implementation is going. Instead of waiting seven or eight years until the first engagement was completed and then receive feedback that it may have worked or could have been better, the Board Committee of Development Effectiveness said, “Please do this early on so that if we need to make course corrects, we need to do that sooner rather than later.”

In what practical ways does IEG help influence the World Bank Group’s operations and decision-making?

It can be hard to trace IEG’s influence and the extent to which it helps the World Bank Group move the needle and change its behavior. I’m the first to admit that I cannot put my finger on it precisely. As the World Bank Group faces questions over whether it learns, and we at IEG are questioned about our effectiveness, we’ll look into how we can prove the case that our evaluations have had a real impact.

In the meantime, let me share a couple of examples where evaluations have been influential.

One very visible example of where IEG has made a difference relates to the World Bank’s safeguards reform, which was partially triggered by an IEG evaluation in 2010 that recommended they be reviewed and updated. Throughout the four-year reform process, we provided feedback on the directions in which management was taking the reforms, based on their own work and the consultations they’d had with stakeholders.

Another example relates to the World Bank’s procurement policy, which represents billions of dollars worth of business. Improvements in this area can directly help developing countries utilize resources more effectively. While the bank worked on revising its policy last year, we conducted an independent evaluation in parallel. Once the bank had revised the policy, they ran it by us to ask whether it addressed all the concerns we raised in our evaluation. It’s an example where timing and close collaboration increased the evaluation’s influence.

These two examples were fully fledged evaluations that were discussed by the board and followed up with a formal action plan and a management action review. But, there are also less visible, more informal ways we influence the bank’s strategy.

For example, whenever the bank starts working on a policy revision or new strategy it’s circulated throughout the Bank Group, including to us. We provide input during these internal consultations so that management can incorporate lessons from IEG as they’re drafting their new strategy. Our evaluations are not necessarily referenced publicly, which makes it hard to trace when we have had an influence.

What are the main findings from IEG’s recent evaluation of the World Bank’s engagement in situations of fragility, conflict and violence?

One key finding is that the bank’s comparative advantage is more on the longer-term development challenges of helping prevent conflict and respond to fragility. For example, if fragility arises from high degrees of inequality, it can help build broader-based economic growth or social safety nets.

A second finding is that quite often the analytical underpinnings — the causes of conflict, the shape of the political economy and the identity of its stakeholders — are either not understood, understood at a very abstract, noncontextualized level, or understood but not translated into a strategy or operational program.

When we talk about economically empowering women, for example, you need to understand each specific group of women — their location, their education, their level of resources, their religion — before you can translate that into strategy and operations. Moving from something as granular as this to something more strategic is admittedly difficult, but we don’t see it happening as systematically as it should.

In low-income, fragile countries, some of which see governments change every six months, we found that the Bank Group needs to be better at building a longer-term vision for countries’ institutions and state capacities in a context that is constantly shifting. In middle-income countries, the story is trickier again, because these countries tend to have very strong institutions. For them, external help may feel like interference with their sovereignty. The Bank Group needs to manage such sensitivities when it works in these contexts.

Ultimately, the World Bank and its clients need to build institutions and capacities in the good times, for the bad times. Once a crisis hits, trying to develop a social safety net is almost impossible because everybody is busy responding to the immediate disaster. It needs to be built in a way that it can be expanded to reach more of the population when needed, with the ability to scale it back when the crisis ends.

How does IEG’s work resonate beyond the World Bank Group? What can the Bank Group learn from others?

I believe that everybody can learn from everybody else. Many of the key lessons from our work in fragility resonate with other institutions. In addition, IEG is often asked in its evaluations to include analyses of the work of regional institutions or development banks. This broadens IEG’s evaluative space, making it almost like a benchmark exercise, and can create a way of learning across institutions.

Unfortunately there is not enough learning.

Thinking back to a previous job, I was involved in a large evaluation of assistance given to South Sudan before it became an independent country. The evaluation found that while political economy analyses were done, they were not used in the forward planning. These analyses pinpointed the exact areas where the most recent conflicts took place. It’s a clear case where learning from both the political economy analyses and the evaluations could have helped take preventive measures, instead of being caught by surprise.

Lessons like this should resonate at the World Bank and beyond and echo the humanitarian sector’s repeated calls to build preparedness capacity before crises hit.

How effective are joined-up approaches between humanitarian and development organizations in fragile states and how is the World Bank supporting this?

This is a tricky question and a long-standing issue.

The on-the-ground emergency assistance that humanitarian organizations such as UNHCR or UNICEF provide, often in the middle of a conflict, requires the ability to react quickly. They have to deploy people rapidly and navigate complicated logistics. Development institutions such as the World Bank, the International Fund for Agricultural Development or regional development banks don’t have that mindset because that’s not what their day-to-day work requires. They’re more focused on development trends and how to make systemic change come about.

While it’s important these two sectors collaborate closely, we must not end up in a situation where those doing more systemic work are asked “to jump onto the truck” and behave like emergency responders and vice versa. That doesn’t draw on the comparative advantage of either party. It’s vital to respect the two sector’s different competencies and ensure each complements the other.

That said, in my previous experience at the World Food Program, I was surprised how separate the humanitarian and development worlds are.

I’m more cautious than optimistic, however, that a renewed commitment to joining these two worlds will be successful. There’s lots of enthusiasm, but I’m not sure how much people are seeking lessons from the past and recognizing their value.

Join the Devex community and access more in-depth analysis, breaking news and business advice — and a host of other services — on international development, humanitarian aid and global health.