How to measure the impact of advocacy

In an era of Trump, Brexit, the global gag rule and increased spending of official development assistance domestically, advocacy campaigns highlighting the benefits and value of investing in development and developing countries are increasingly important. It is not just an era of stagnated politics, but fake news and targeted news, with social media responsible for telling people at both extremes of politics what they want to hear. Making inroads into these groups can be very difficult. But for advocacy campaigns to make a bigger impact, experts are urging that strategies be developed as part of the campaign to include measurements of impact. And by sharing these outcomes among the development community, future campaigns can avoid what did not work, expand on what did and be better prepared for barriers to making an impact. Why is it important to measure the impact of advocacy? Measuring the impact of advocacy, experts told Devex, has a range of important benefits — from justifying campaigns, supporting supporters and ensuring assumptions of the initial program of advocacy are correct. “One of the most important reasons to measure impact is to ensure that what you’re doing is making a difference and creating the change that you set out to achieve,” Tony Milne, director of the Campaign for Australian Aid, told Devex. “In social change movements, it is critical that your supporters and volunteers know that their time and energy is helping to make steps, even if sometimes small steps, towards change.” When measuring impact, it is also important to look at unpredictable external influences and their ability to sway a campaign. “Policy and programmatic changes, norm shifts, court decisions and so on can have unintended consequences,” Jennifer Dreher, program director for Geneva Global, told Devex. “It is important to understand that the assumptions of your advocacy hold true, and that an advocacy win had the intended impact on its target population. It is just as important to measure the impact of when advocacy has been able to prevent a setback, as it is to know that advocacy has advanced an issue forward.” But it is also a way to better understand decision making and the mindset of those whose opinions need to be swayed. “Policy advocacy is aimed to change the mid of policy makers,” Matt Frazier, partner and advocacy expert for Dalberg, explained to Devex. “What we have learned over the past several decades is that it is hard to understand how our own minds get changed. We need ways to dissect the decision framework of policy makers.” How do you measure the impact of advocacy? There is no simple formula to measuring the impact of an advocacy campaign — it is dependent on the issue, audience and method of seeking to influence people and groups. “One of the things that makes advocacy tricky is that it is tailored to a particular market and requires a real understanding of both the political economy of the government you are targeting and also the specific policy agenda associated with the campaign,” Frazer said. Different actors within the development sector can play different roles in the advocacy community — some organizations specialize in short-term policy changes while others can focus on long-term social change. Christa Lane Hooper, a senior associate with Geneva Global, agreed, saying measuring impact first begins with understanding what advocacy is setting out to achieve. “When trying to accomplish systems change, it’s necessary to consider what role advocacy plays,” she said. “For example, to change societal norms around an issue, such as education for women and girls, you need to work on each layer of the system — from policymakers to civil society organizations to influential public figures, all the way to local communities.” Advice from Geneva Global is that in beginning the process of measuring the impact of advocacy, it is important to understand what the organization can control and influence, and what falls outside of their control to set achievable, project-specific benchmarks. “If you make small mistakes quickly, learn from them and improve. Adjust the strategy if necessary, but also don’t give up or compromise your vision too quickly.” --— Tony Milne, director of the Campaign for Australian Aid And with this, Milne says, the benchmarks should be aiming to keep it simple — what has changed, how do you know it’s changed, how do you know it is your work that has caused that change to happen? “Have a clear definition of victory or success, and work backwards from there, including developing key performance indicators that help take you forward,” he advised. Those KPIs should be clearly communicated to everyone associated with the campaign to ensure everyone has a similar measurement of success. It is important to evaluate and measure as you go, Milne said, not simply at the end of a campaign. “If you make small mistakes quickly, learn from them and improve,” he said. “Adjust the strategy if necessary, but also don’t give up or compromise your vision too quickly.” “Celebrate the small wins as well as the bigger ones,” Milne continued. It is not just important in measuring impact but in encouraging continued action to impact development issues. Measuring the impact of policy should factor in long-term monitoring, and looking at the capability within the organization or group advocating for an issue, Frazier advised. “The sector has erred, in some ways, on focusing on intermediate and short-term outputs as evidence they are heading down an appropriate path,” he said. “Everyone in the community recognizes that policy change can sometimes take years or decades. Chasing after six-monthly evidence of progress is attractive but it assumes a linear model of change.” Advocacy successes, Frazier explained, progress slowly before a final burst leading to impact. The question then is what to focus on. Tools and frameworks can be used to tell an organization how effective it is by monitoring progress in three different areas: How the organization is positioned strategically in relation to the policy change they are advocating for, the advocacy strategies selected to achieve the change and how effective is the method selected. “Most organizations spend 90 to 100 percent of their time on the third part, tactical effectiveness,” Frazier said. “If they were to focus more of the first and second components of this framework they will more confidently be providing a truthful measure of the impact of their advocacy.” Case study — the Campaign for Australian Aid The Campaign for Australian Aid’s broad mission is to create a public movement around an expression of Australia’s place in the world and common humanity, leading to increased political support for an effective aid program. It has three broad areas that form its “theory of change.” First, to generate active public support for achieving the vision through powerful grassroots presence in the real world. Second, to leverage public support in creating political space for progress, with core supporters helping to win the aid debate. And third, to engage one-on-one with politicians to build deep political support, for the medium- to long-term. Monitoring the impact of their advocacy includes assessing the campaign over a three year period — three year KPIs that sit below each of their theories of change and the 12-month plan breaks that down further. “All of which align with our broader vision,” Milne told Devex. “Our strategy for the next three years was informed by an external evaluation of the previous three years, and an analysis of the broader global and domestic political context and the risks and opportunities of that context.” The analysis of the impact of their campaign against their three KPIs showed massive achievements. “Over the past three years, we’ve built a campaign from a few thousand supporters, to almost 145,000 people, alongside a re-engagement of churches and youth in campaigning to end global poverty,” Milne said. A growing grassroots presence has improved their capacity to respond to external events and to advocate for the aid program. But they are also measuring impact through collaborations between individual aid agencies. “It’s the often invisible side of campaigning but includes the building of trust within and between organisations, the building of social capital and relationships, the sharing of information and data and co-operation around joint messaging and joint moments,” Milne explained. While support and capability have improved, an increasing aid program budget — the other measure for their advocacy campaign — has continued to decline. “We will need to use the solid foundation we’ve built to take the campaign to the next level,” Milne said. Barriers to measuring impact The biggest failures in advocacy, Milne said, come when there is no clear definition of what victory or success looks like, or everyone has a different idea of what victory or success looks like. “Other failures can be if there isn’t a shared understanding of the ‘theory of change’ or understood norms and ways of working,” he said. “Internal division and conflict are the often a greater barrier to success than external factors.” Similarly, Geneva Global has found that overly broad project goals can also be barriers. Through their work, they have found that that partners who outlined only broad project goals were challenged to determine how tactics contributed to their results. Their advice is to have specific goals help define what the project seeks to accomplish, providing a clear target for activities. Frazier believes NGOs need to avoid looking to measure impact on areas outside of their control. “If advocacy organizations worry about the impact the Global Fund has on the ground, for instance, this is not something they are well equipped to do,” he said. “My advice is to focus on your strengths.” Measuring impact has wider impact than a single campaign “Organizations need to understand their role in the landscape and how they fit into the larger ecosystem to ensure that efforts aren't being duplicated.” --— Dakota Dobyns, consultant with Geneva Global Advocacy in the current political climate requires collaboration, explained Dakota Dobyns, a consultant with Geneva Global. And NGOs need to avoid competing to better understand impact and create change. “With an influx of funding, it is an opportunity to collectively bring about the change you seek,” Dobyns told Devex. “It’s the time to build coalitions and collaborate to truly move the needle on an issue. Organizations need to understand their role in the landscape and how they fit into the larger ecosystem to ensure that efforts aren't being duplicated, but instead they're working with partners to leverage each other's strengths and efforts.” Sharing information on the impact of an advocacy campaign — including audiences targeted, mediums used to communicate a message and external impacts — is part of this collaborative effort to continue evolving advocacy campaigns and making strong inroads into hard-to-target groups. “Measuring the impact of advocacy does not only benefit the project being implemented, but it also helps leaders and policymakers understand the level of influence you’ve achieved, which can help build them as champions for your issue,” Hooper said. Done right, the result should be evolving campaigns of advocacy, growing from campaigns that came before them, which continually increase global support for aid and development to help those in need — regardless of age, sex and religion. Read more international development news online, and subscribe to The Development Newswire to receive the latest from the world’s leading donors and decision-makers — emailed to you free every business day.