How results-based financing failed to live up to the hype

It's 2010 and aid donors are on the hunt for new ideas. Billions of dollars are being poured into aid but the results are often unclear and governments are facing increasing pressure to prove the value of aid to skeptical taxpayers. A plan gains traction to shift the focus from inputs to outcomes. Instead of paying for a hospital to be built or healthcare professionals to be trained, the donor would only stump up the cash when, for example, child stunting or maternal mortality rates fell; instead of funding new schools and teaching equipment, they'd pay for an increase in literacy rates. In return, donors would step back and let the implementers figure out how to achieve those results — fostering innovation, localization, and building institutional capacity. The concept, and related jargon, snowballed. Results-based financing, cash on delivery, payment by results — whatever you prefer to call it, by 2017 an estimated $25 billion of development money had been tied to results. New financial instruments and funds also emerged to deliver this new type of financing, including development impact bonds, the World Bank’s program for results, and outcomes funds. But nearly 15 years later, have results-based approaches delivered on their transformative promise? A brief history Results-based aid emerged in a climate of disillusionment with development. Approaches to aid had changed over the decades but by the 1990s, concerns about corruption, wasted funding, and aid dependency were rife, leaving the sector looking for new approaches. While the idea of paying for results is centuries old, it didn’t become mainstream in development until around 2010, helped by the work of Nancy Birdsall and Bill Savedoff from the Center for Global Development who coined the term “cash on delivery” to describe their blueprint for transforming the aid sector. “We started to realize that if the public health system is messed up, it doesn’t help to just build more hospitals. A more innovative approach is needed,” Savedoff told Devex. “Results-based approaches get people to remember that when you’re designing a program it’s not just to build a hospital, you’re designing a program to make people healthier.” In its purest form, “cash on delivery” was conceptualized as a transfer of power, responsibility, and decision-making from the donor to a recipient government, Savedoff explained. He had hoped for broad programs — a goal might be set around energy access, for example, with the recipient government free to engage with whomever might help the most, and money flowing from the donor in relation to how much things improve. Despite the promise of this radical new approach, it proved difficult to deliver in practice. Donors are often unable — politically or bureaucratically — to relinquish control to the degree required, and so results-based financing has often been implemented in ways that strip it of its key benefits, experts told Devex. ‘The worst of both worlds’ The U.K.’s now-defunct Department for International Development, or DFID, which was one of the first to pilot results-based finance, is a good example. It launched education pilots in Rwanda and Ethiopia in 2012, which were later expanded through the Global Education Challenge — a £355 million (about $428 million) program to boost girls’ education across 18 countries. GEC was mostly structured as a traditional aid program, according to Paul Atherton, who was an education economist for DFID at the time. But for some of the projects, DFID made 10% of funding contingent on achieving pre-agreed outcomes around attendance and learning levels. If the outcomes were not met then the implementing NGOs lost money; if targets were surpassed, they got a bonus. The idea of the 10% cap was partly to avoid passing on the full risk of a new methodology to partners, Atherton explained. But this hybrid approach made it impossible for DFID to be truly “hands off” and leave room for innovation and localization. With 90% of the project paid on inputs, it was subject to the normal donor tracking and reporting mechanisms. At the same time, the results-based portion meant it needed rigorous monitoring and evaluation. From the implementer's perspective, “it means you’ve got the worst of both worlds — bureaucratic reporting and risk,” Atherton said. For Paul Clist, a development economist at the University of East Anglia, the DFID experience proves that results-based financing doesn’t work as an add-on to a traditional project. The final evaluation reported no upside to the results-based portion of the project and found that, far from fostering innovation, it made the implementing NGOs take a “safety-first approach” to ensure they didn’t lose funding. GEC also threw up verification challenges, according to the head of one of its implementing NGOs, who asked to remain anonymous to maintain professional ties. Implementers felt that the way results were evaluated was crude and inaccurate. For example, students were given “one-size-fits-all numeracy and literacy tests” to measure improvement, but since the rate of improvement naturally slows at the secondary school level, implementers managing secondary-level programs lost out on payments, the NGO boss said. Outside of GEC, there have even been cases of implementers gaming results to hit targets. In response, donors need to design “context-relevant outcome measures that will be assessed collectively to determine the degree of performance,” the NGO boss recommended. But the bigger problem, experts said, is that for results-based financing to work, it has to be done at much higher risk levels in order to properly shift the incentives — removing limits such as GEC’s 10% safety cap. This is difficult for donors, who need to be able to justify their aid spending over short time spans and often can’t be truly hands-off. As a result, pure results-based financing has never really been tried, according to Clist. “It’s easy to blame donors for not implementing a good think tank idea, but I don’t think cash-on-delivery aid thought deeply about the constraints that donors face,” he said. The model also fails to address the constraints facing small and local NGOs that are likely unable to shoulder the financial risk and complexities of results-based contracting and end up being excluded, according to the NGO boss. Go big or go home As a result of some of these challenges, much results-based aid is now done through specialized outfits that have the expertise to design contracts and measure outcomes. The biggest specialist funder is the World Bank’s “Program for Results” instrument, launched in 2012. In 2022, PforR lending accounted for $10 billion, approximately 15% of the bank’s overall lending, a spokesperson told Devex. The bank’s evaluations show that “PforR” projects are delivering their agreed results. Demand has also grown among client countries, who value the unrestricted budget support, and the bank’s shareholders were recently convinced to lift the limit on results-based lending. But its success all depends on how you define “results”. For purists, results relate to the impact of a project, such as increased learning outcomes for an education project. The bank, however, takes a broader approach, paying out for simple outputs such as designing a student scholarship scheme, regardless of the impact it has. A 2014 CGD review of the bank’s first 35 PforR programs found that only 8% of the lending was connected to true outcomes. The rest of the funding was tied to actions, processes, or outputs, such as the preparation of an environmental guide or a number of new bridges built. The same issue was flagged in a 2014 progress report by the bank’s internal watchdog. According to Joseph Holden, an independent evaluation specialist who has worked on PforR projects, the process-related yet often complex nature of indicators led to a “target obsession” among implementers and a relentless focus on reporting that left little room for innovation and flexibility. There is a reason for this, however. True outcomes can be difficult to cost out and track. They may be out of the direct control of implementers and slow to emerge, making it difficult to use them as part of a contract, according to Amanda Glassman, executive vice president of CGD. Birdsall and Savedoff had envisioned 30-year timelines for results but donors rarely offer more than three years of funding. “When a government is negotiating with the bank, it wants [to be paid for] stuff it can control,” she explained. ‘Good enough’ Although results-based financing has mostly only been implemented in partial ways, there have still been some benefits, experts said. For Glassman, paying for outputs is a step in the right direction. “If the bank is paying for outputs further up the chain, such as the number of antenatal visits, then I think that’s good enough because we know it’s connected to outcomes,” she told Devex. She also said that the bank’s PforR work has led to much better monitoring and evaluation, both during implementation and afterward, which can only be good for the sector. Savedoff agrees that PforR is a far cry from his cash-on-delivery vision, but he does see positives, including greater ownership of programs for client governments and more collaboration between the bank and government officials. But the model still hasn’t really taken off outside of a few multilateral programs, Glassman said, and many experts believe there is still much more experimenting to be done. For some, the future of results-based aid may even lie outside donors. Avnish Gungadurdoss, the founder of results-specialist Instiglio, wants to see it used primarily by governments to improve public policy in their own countries, turning recipient governments from implementers into results-based funders themselves. “I don’t think results-based aid has really been tested or evaluated enough yet,” Glassman said.