Are development organizations failing to learn from failure?

When the misuse of millions of dollars disbursed by the Global Fund to Fight Aids, Tuberculosis and Malaria in several countries became public in 2010, media stories denounced the misappropriation of taxpayers’ money. They concluded that such instances of fraud likely undermined the credibility of the fund, leading several donor countries to reassess their contributions. GiveWell, the nonprofit that rates charities by assessing their impact, disagreed. What the media failed to appreciate, Co-Founder and Co-Executive Director Holden Karnofsky wrote, was that the fraud had been disclosed by the Global Fund itself, in a transparency effort of a scope rarely seen in the development industry. “In our investigations of large charities, we’ve seen no evidence that donors are auditing them in a way that would force disclosure of lost funds,” he wrote. “We’ve also seen no evidence that these organizations have any way of even knowing when funds have been misused.” For its effort, the Global Fund deserved “praise, not hostile headlines,” he added. Media stories about ineffective aid programs and wasted development money are plentiful, while the supposed failure of the aid sector as a whole has been the subject of several books and academic articles. The stigma attached to failure has become so great that few charities will ever publicly admit that one or several of their projects hasn’t done well, and even fewer will take responsibility for it. Yet a growing number of professionals in the aid industry are trying to break the taboo, saying admitting failure and learning from it is an inherent part of becoming more effective. “The appalling part of the international development industry is the repetition of things that don't work. That to me is unacceptable; that’s the ultimate failure.” --— Ashley Good, founder and CEO, Fail Forward GiveWell has consistently called on charities to be more transparent about their flaws and mistakes; one of the best ways for an organization to include its rating on the platform is to publicly demonstrate failure. Karnofsky has even gone so far as to suggest a grant idea that would help nonprofits prove their programs don’t work (GiveWell regularly publishes reports on its own mistakes.) Engineers Without Borders Canada has also made failure an integral part of its culture. Since 2008, the organization has published yearly “failure reports” in which it addresses its projects’ shortcomings. David Damberger, who founded EWB Canada’s Calgary chapter, gave an enlightening TEDx Talk where he reflected on his responsibility in letting a project fail. His colleague Ashley Good, after working with EWB Canada on a project in Ghana that failed year after year, went on to create Admitting Failure, a platform on which nonprofits can share their own stories of failure in the hope that others can learn from their mistakes. By her own admission, the platform has failed, in spite of earning praise from the sector, because not enough organizations dared to come out publicly about faulty projects. Good has since moved on and founded Fail Forward, which helps businesses, funders and nonprofits learn from failure. And why would organizations admit failure? There are currently few incentives or rewards for implementing organizations to do so. Effectiveness is measured by results achieved, objectives are agreed upon early on, and funding renewal depends on a project’s previous track record. “Accountability tends to be to donors and senders as opposed to beneficiaries,” Good told Devex. “In the private sector, if you’re failing, you either fix it or you go out of business, because your customers have the power of sanction by not buying your product. Beneficiaries in international development do not have that same power.” Donor agencies struggle themselves to learn from their own mistakes. A recent report by U.K. aid watchdog Independent Commission for Aid Impact concluded: “The emphasis on results can lead to a bias to the positive. Learning from both success and failure should be systematically encouraged.” The overall emphasis on positive results has been detrimental to the sector as a whole. Caroline Fiennes, director of Giving Evidence, suspects most research produced by U.K. charities might be unpublished. Withholding research might help organizations highlight the good work they’ve done, but it also leads to bad projects being replicated again and again if the evidence against them is never made public. Ultimately, opening up to the possibility of failure isn’t the responsibility of organizations alone, but will likely require a greater paradigm shift in the sector. So how can we make it happen? 1. Think like a scientist Science can teach us a lot about failure. After all, scientific progress doesn’t happen overnight, but is often the result of numerous failed experiments. Scientists know how to extract valuable information from both failed and successful projects to create new standards and products. Randomized controlled trials are a great example of how the development industry can use scientific evidence about what doesn’t work to set up successful interventions. 2. Learn from entrepreneurs Consider this: Only 10 percent of new products ever succeed on the U.S. market, while 25 percent of startups fail in their first year. Of those remaining, 36 percent fail in their second year, and the survival rate only grows to 50 percent in their fifth year. Yet entrepreneurship is consistently considered a pillar of the economy, in spite engulfing millions of dollars in lost venture capital each year. In recent years, the business world has accepted failure as a path toward success, dedicating entire conferences to the subject. Certain experts have come up with tools that could equally be used by development professionals. Harvard’s Amy C. Edmonson has devised a “Spectrum of Reasons for Failure,” which helps organizations make the difference between “intelligent failures” (e.g., failing when experimenting) and “blameworthy” ones (e.g., deviation from a prescribed practice.) 3. Build a culture of learning and innovation Learning from both failures and successes goes hand-in-hand with innovation. Development organizations should be encouraged to dedicate part of their activities to experimentation, perhaps through pilot projects, without suffering any negative consequence from inconclusive results. This might imply having discussions about the ethics of experimentation when livelihoods are at play. Engaging all employees in the learning process is critical to detect failure at all levels. Yet there are many reasons why employees might refrain from speaking up, from fear of blame to lacking a platform to voice their concerns, to knowing the information will not be taken into consideration. To encourage participation, organizations can set up mechanisms that allow for productive feedback (e.g., monthly meetings in which employees know they can speak freely, workshops, etc.) as well as processes to evaluate feedback and eventually translate it into change. 4. Analyze failure Failure can be complex. More often than not, it is the result of a series of events and a combination of factors. “We tend to attribute successes to our own brilliance, and failures to external factors, which is absurd,” Good said. When analyzing failure, organizations too often choose to blame others, leading to what is called a “fundamental attribution error.” Good thinks most projects should be expected to contain elements of successes and failures. What matters is to look at what went right and wrong, and why it happened. Another type of failure often goes unnoticed, according to Good, and that’s the failure of inaction, or when we’ve failed to take action when we should have. 5. Learn from failure and build resilience Detecting and analyzing failure isn’t the end of the process — learning from failure includes translating lessons learned into change. “The appalling part of the international development industry is the repetition of things that don't work,” Good told Devex. “That to me is unacceptable; that’s the ultimate failure.” But just like entrepreneurs, development professionals can develop “loss aversion” — when we focus too much on the pain associated with failure, and prefer avoiding losses rather than acquiring gains. Seeing failure as a learning opportunity helps build resilience and concentrate on the bigger picture. 6. Share your results Fiennes of Giving Evidence thinks one of the reasons charities withhold research they’ve produced themselves is there is nowhere to publish it. “There are few repositories or journals, nor standard ways of ‘tagging’ research online to make it findable, so charities may (rightly) think that the traffic to material just on their own websites won’t justify the work in sprucing up the research to publish it,” she wrote. While some organizations like EWB Canada and One Acre Fund have taken to publishing failure analyses on their websites, most research for what works and what doesn’t, as well as postmortem reports for completed projects, remain hidden or dispersed. Sharing results, good and bad, would help prevent ineffective interventions from being replicated. What types of incentives do you think would help encourage organizations to be more open about their failures so that they — and others — can learn from them? Let us know by leaving a comment below. Check out more insights and analysis for global development leaders like you, and sign up as an Executive Member to receive the information you need for your organization to thrive.