Will travel restrictions transform monitoring and evaluation?

BURLINGTON, Vt. — The pandemic has created new rules for international travel, and the global development industry has not been spared. While restrictions caused by the COVID-19 pandemic have created significant disruption across the health, development, and humanitarian landscape — interrupting a range of basic services, and threatening some long-held goals — they have also created an opportunity for introspection and could serve as a catalyst for changes that many agree are overdue, experts told Devex. “I think there are a lot of things in the industry that we’ve said for years about local capacity building, local agency, using local partners, and moving away from a completely expat-driven model, that — I hope — this is forcing the hand on,” said Keith Ives, co-founder and CEO of monitoring and evaluation firm Causal Design. Professionals working in monitoring, evaluation, and learning are changing cost-benefit analyses, accelerating shifts towards flexible and adaptive management, and creating new opportunities to leverage local leadership and existing data. ‘Should we be doing this?’ Global development donors have largely avoided blanket restrictions or guidelines related to travel and project implementation during the pandemic, and the experts who spoke to Devex said that is a welcome approach, even if it does create challenges. “What we’ve seen is a lot of understanding and flexibility of what’s possible in this environment,” said Barbara Willett, interim director of monitoring, evaluation, and learning at Mercy Corps. That flexibility is part of an ongoing shift toward more adaptive management, which seeks to allow organizations to learn and shift their operations over the course of a project, she added. “We expect to adapt. We expect to be learning, and observing, and monitoring ... That’s just sort of built into our mechanisms already,” Willett said. The COVID-19 pandemic is testing that flexibility, however, as different countries adopt different travel restrictions and guidance on a near daily basis. What might be an irresponsible risk in one country could be perfectly feasible in another. “You can’t make blanket guidance. We can’t just say, ‘no in person surveys,’ because that doesn’t make sense,” Ives said. “If we put fewer expats on planes ... and we actually invest and care about the capacity of our local staff, because we can’t justify spending 30% of our budgets on travel anymore, that’s a good thing.” --— Keith Ives, co-founder and CEO, Causal Design In response to a question from Devex about what guidance the U.S. Agency for International Development has given to organizations working on monitoring and evaluation about how they should approach decisions about travel, a USAID official wrote that the agency is “working with implementing partners to find innovative, responsible, and safe ways to monitor and evaluate programming.” The official added that if partners are experiencing disruptions to the implementation of a program, they have been instructed to inform their counterparts at the agency. “To USAID’s credit — and I think this is what’s caused a lot of M&E efforts to slow down and screech to a halt — is that they’re having to have these conversations on a case-by-case basis,” Ives said. The challenge for donors is to find a balance between guidance and flexibility that encourages their implementing partners to move from asking how to carry out activities during a time of COVID-imposed restrictions, to asking whether it makes sense to carry out those activities at all, Ives said. He described a meeting in which a competing firm “frustratingly” sought to reassure the donor that they would be able to move forward with a planned survey while social distancing, while Ives argued that doing so simply did not meet a basic cost-benefit calculus in the midst of an infectious disease threat. “This is a moment in time where knowing the results of our panel study on food security just isn’t worth the risk of infecting a single person,” he said. Leveraging what exists At a time when countries are desperate for granular data to guide decisions about whether to impose lockdowns, close or open schools, or restore economic activity, firms that help generate that data have a big responsibility to ensure it is as useful as possible. That does not always mean funding a new project with its own bespoke indicators and data collection requirements, but instead focusing on putting existing local data to better use, said Hannah Klein, CEO and co-founder of Cooper/Smith, a firm that helps governments use data to improve health and development outcomes. “There is a ton of data that’s already available in-country, but it’s just not really being leveraged or used,” Klein said, adding that what is needed is “data janitorial work — cleaning the data, relating the data, and really thinking about how it can be used and brought together.” “COVID has sort of pushed that forward — leveraging data that are already out there rather than collecting new information, and really starting from the point of view of understanding the key questions and decisions that ministry officials are trying to make at this time,” she said. Klein described the enormous burden that healthcare workers and others face when it comes to data collection for health and development projects, citing research that found 3,500 different indicators related to HIV programs in Malawi. “Oftentimes it’s overkill, it’s duplication, it’s not being streamlined, so people are spending a huge amount of time doing collection of data, but they’re not using it,” she said. Local talent, local leadership The global health and development community has a similar opportunity in this crisis to follow through on its long-held commitment to putting more power and decision making in the hands of people closest to the problems they strive to solve, experts told Devex. “If we put fewer expats on planes, and we actually rely on our local partners, and we actually invest and care about the capacity of our local staff because we can’t justify spending 30% of our budgets on travel anymore, that’s a good thing,” said Ives. The demands and restrictions imposed by COVID-19 could help knock down some of the structural barriers that have stood between local professionals and leadership positions, Ives hopes. “There’s always these masked ways that I think we discriminate against local nationals — how is their writing, how are their presentation skills, will they be good in front of the donor — these sort of things that I think, in the context of our social crisis in the U.S., we would say is a form of sort of institutionalized racism or discrimination,” Ives said. The COVID-19 crisis, he hopes, might create a moment to examine the opportunity cost of staffing evaluation teams with international experts, versus local national talent. He said that clients tend to say they want both international caliber evaluation plans and investments in local capacity. “My experience has been that they’ve always chosen more rigorous and robust statistical models, [rather] than more investment in local human capital for the project,” Ives said. “I think we’re taking some of that off the table right now because of COVID, and I think that’s a good thing,” he added.