Going for the gold: Do's and don'ts in globaldev rankings

Just as nations compete for top spot in the 2016 Summer Olympics medal tables this month, their performance is ever under scrutiny when it comes to development. Nonprofit organizations often rank the best and worst performing countries in a specific area. How can these bodies make the most of tables, scorecards or indices to help achieve their aims? Development professionals use rankings in many ways, but often cite their usefulness as a media tool. A breakdown of country level results can empower local advocates and media to highlight their country’s performance or need. But achieving a robust index can be a complex and time-consuming task. And development organizations don’t always compare only countries — “naming and shaming” other entities, such as businesses, sectors or even technologies can bring them closer to achieving development aims. Do focus on an objective Presenting data as a scorecard can refocus attention on a development issue. Last month, U.K. charity WaterAid published a report ranking countries by prevalence of stunting among children. “Caught Short” named India as having the greatest number of children who were stunted. Jo Lehmann, the report’s author and WaterAid senior media officer, used data contained in the 2016 Global Nutrition Report, released in June, to compile the comparison of 132 countries. “That report was so vast and does not focus specifically on stunting and water, sanitation and hygiene — we wanted to highlight in closer detail the worst places in the world for stunting,” Lehmann told Devex. Presenting the information as a comparative table, Lehmann suggests, is more attractive to media organizations as it lends itself to strong headlines. “We’ve found in the past that by honing in on one issue and ranking it, it tends to get more media pick-up, the idea being that if we can get pick-up in the mainstream press it will filter through and have the result of influencing policymakers,” she explained. Do localize results Breaking down country-level results can empower local advocates and media to highlight their country’s performance — or a lack thereof. Save the Children USA has published a State of the World’s Mothers report annually for 16 years. This compares what it’s like to be a mother and raise a child in more than 160 developing and developed countries. Scoring countries on factors such as risk of maternal death, under-5 mortality rate and participation of women in government, the 2015 report named Somalia as the lowest-performing country and Norway the highest. Wendy Christian, associate vice president for marketing and communications at Save the Children, told Devex the index was originally developed as a media tool. “An index helps an international organization have a media angle for each of its country offices,” she told Devex. “Media is more hyperlocal now than ever before so an index can give us an opportunity to tell a global development story in a way that is relatable and feels local to media and the general public in markets all around the world.” By naming countries, Christian said the report could also be used as an advocacy tool. “It was a calling card for our staff to be able to go in and sit down with people who could really make a difference to mothers in each country and talk about the issues,” she said. Christian further explained that even when a country appeared at the bottom of a ranking, this could be turned into a positive opportunity for the charity to emphasize the existing needs, and how they and those responsible could work together as partners to address them. Be thoughtful, don’t oversimplify the data or the problem Compiling a ranking is never a simple task. It may have serious implications for the countries it lists, so the data must be academically robust. For this reason, Save the Children commissions a university research team to conduct its research each year. Even so, Christian warned there could be shortfalls in data when relying on in-country information. “The government might not have the same resources or stability, for example, if there’s been a civil war in that country,” she said. Christian also advised development professionals to prepare partners in countries or sectors that appear in a ranking in advance of its publication. She advised in-country staff to meet, for example, with health and education ministers if their country scores poorly on an index. “I understand what it’s like to be on the low end of an index,” said Christian, who is American. “The U.S. was at the bottom of developed countries for mothers.” Save the Children’s index, Christian explained, was further a method of measuring their programs’ success in a country. Having worked with local leaders to make improvements in, for example, access to midwives, it was a way of demonstrating where efforts and investment had paid off over time. “We thought that was important from a storytelling perspective, so that we could say to member of the general public or policymakers that progress does happen,” she added. Create positive competition, and repeat Countries are not the only entities that might be driven to make improvements by public comparisons of their achievements. Oxfam GB has used indexing to compare how food and drinks companies’ policies affect issues such as land rights, climate change or farm workers’ incomes in developing countries since 2013. By scoring companies including Unilever and Nestle at least annually, it generates a table of the best and least performing organizations. The charity’s head of private sector team Erinch Sahan told Devex the approach aimed to empower the public and consumers by giving them information to hold companies to account, and “trigger a race to the top.” “It shook up the sector quite a bit,” Sahan said of the “Behind the Brands” campaign. “We ended up with over 700,000 consumer actions taken over the couple of years of campaigning, such as people tweeting companies to say ‘we're disappointed you're not doing enough on this,’ or signing petitions or jumping on their Facebook page.” Sahan stresses organizations using indexing tools should repeat the exercise to show improvements over time. “If you do it once and go away, it’s very easy to ignore,” he said. Since its inception, Oxfam has had to change the indicators it uses to assess the brands as the companies began to behave differently. Sahan said this is another important factor to consider. “They were doing things beyond the indicator. When that happened we updated the indicators and gave companies additional credit,” he said. “A couple of years in, we noticed some companies started looking at what percentage of the value of the product was going to farmers. We thought this was something we could reward them on.” Sahan said it was important to engage with the companies and give them an opportunity to correct any information throughout the process. “It’s onerous to do an index well,” he suggests. “But it’s a useful way of packaging communications with the public and a good framing for public discussion.” Sahan adds that countries, as well as companies, can respond to rankings competitively. He cited the World Bank’s “Doing Business” report, which compares business regulations in 189 economies, as one example. “Countries do want to see themselves progress,” he said. As the world ranks its athletes this month in Rio de Janeiro, organizations looking to raise the profile of global development issues might consider what their version of gold, silver and bronze could look like, and whether competition can drive unexpected results. Check out more insights and analysis for global development leaders like you, and sign up as an Executive Member to receive the information you need for your organization to thrive.