Taking the SDGs on a road test

As the international development community works to define the post-2015 agenda, many questions arise regarding the feasibility of measuring progress toward the sustainable development goals: How big are the data gaps, and how can we fill them? How can we make sure that marginalized communities are being accounted for? What does the universality of SDGs mean when applied to local contexts? Is there a place for citizen-generated data?

For a group of researchers from the Center for Policy Dialogue, Normal Paterson School of International Affairs and Southern Voice on Post-MDG International Development Goals, the answer to these questions was simple: Let’s road test the SDGs and see what happens.

With funding from the International Development Research Center’s Think Tank Initiative and the Hewlett Foundation, and with support from the U.N. Foundation, researchers partnered with local think tanks to test seven goal areas and about a dozen SDGs in Canada, Bangladesh, Ghana, Peru, Senegal, Sierra Leone, Tanzania and Turkey. The idea was to get a clearer picture of how the SDGs would apply to a variety of countries, and include southern perspectives to the debate.

“Because this was done by people who work at think tanks at the country level, we’ve been able to go beyond technical discussions around data availability and data quality to really try and unpack the political economy drivers of what's going on around data,” Kate Higgins, manager of the DataShift at Civicus and one of the project’s leaders, told Devex. The Post-2015 Data Test was able to release country reports for Turkey and Canada earlier this year, and will be rolling out other case studies over the summer.

Data gaps

The project has predictably revealed significant data gaps, especially in low-income countries. The biggest challenges were around goal areas that weren’t part of the Millennium Development Goals, such as environmental sustainability and disaster resilience.

But when researchers started looking at ways the missing data could possibly be collected, they found the solution was often within reach. In some instances, a question could be added to an existing survey; other times the data existed but was not accessible.

“I think there are creative ways we need to think about filling data gaps, and there is a base from which we can build. It’s not like there's absolutely nothing there,” Higgins said.

Capacity gaps

The Data Test also showed huge capacity gaps across countries.

Shannon Kindornay, an adjunct research professor at NPSIA, told Devex many ministries and national statistical offices simply don’t have the proper human resource of infrastructure to meet their commitments to the Paris21 agenda for the data revolution. She added the process of measuring progress toward SDGs would inevitably run into the question of “good enough data” — when the data exists but is too old or of poor quality. Kindornay worries the international development community will be tempted to fill these data gaps with technology enabled by external funders, rather than tackling the difficult challenge of capacity building.

“My concern is that with all the excitement around the SDGs, and all the excitement around data, technology and all these kinds of innovations, that we’re going to miss the hard pieces of data infrastructure and statistical capacity that are needed at the country level,” she told Devex. One of the project’s main recommendations, she said, was that governments allocate a greater share of their national budgets to data collection.

Official versus unofficial data

Another way to fill data gaps would be to go to unofficial sources of data, including data produced by civil society, academia or the private sector.

While the post-2015 Data Test focused on official sources of data, it looked at opportunities in each country to strengthen ties with unofficial sources. In certain areas such as infrastructure and energy, Kindornay said, private sector sources could easily fill the gaps. But governments often lacked the ability to validate information coming from nongovernmental organizations, think tanks or academics. National statistics offices should work on developing coordination mechanisms between these different sources, she added.

Higgins, whose DataShift initiative supports civil society organizations to produce citizen-generated data to monitor development progress, said she sees a role for these types of data collection tools in the post-2015 agenda.

“Citizen-generated data typically gives you a much more localized perspective on what’s happening, it gives you a much more direct reflection of what citizens, people and individuals are experiencing. It’s often much more real time, so the information is available in a much more timely manner,” she told Devex. But while the question of using citizen-generated data often gets framed in terms of comparability, Higgins said we could also look at it as a source of complementary data.

“What we’re doing is not just monitoring progress,” she said. “The SDGs are about much more than that, [they’re] about understanding where things have worked, understanding where things haven't, understanding what sort of impact policy experiments have had.” DataShift and ONE Campaign have each recently launched platforms aimed at cataloguing citizen-generated data initiatives around the SDGs.

Leaving no one behind

Draft documents for the post-2015 agenda clearly underline the need to take marginalized populations into account, but the ability for countries to gather good-quality disaggregated data has yet to be proven. The Data Test looked at five types of disaggregated data: location, sex, age, minority grouping and income level.

“What we found is that if you have data on sex, particularly for the MDG-like areas, you can almost always get disaggregation by sex and by location, and sometimes age as well,” Kindornay said. “But generally speaking, if you want to start talking about who is benefiting from progress in terms of income quintiles or … different minority groups that are being left behind, we don’t have that kind of disaggregation.” She added that getting information on multiple forms of disaggregation, such as looking at middle-income minority groups, was even more difficult.

According to Kindornay and Higgins, one of the main challenges around collecting disaggregated data has to do with the political dimension of data, as marginalized groups can be unaccounted for by national governments, or their status as a minority is left unidentified for historical reasons — many countries leave ethnicity out of their national census, for instance.

Both stressed that part of the conversation around how to implement and measure SDGs will have to address the political context surrounding data in a strategic way. Kindornay added this issue can come up in developed countries as well. Canada, for instance, doesn’t always do a good job of accounting for First Nations people in general surveys.

“As much as we say who gets measured and what gets measured is a political decision in the [global] south, it’s a political decision here too,” she explained.

Read more development aid news online, and subscribe to The Development Newswire to receive top international development headlines from the world’s leading donors, news sources and opinion leaders — emailed to you FREE every business day.