Who will lead development's big data revolution?

In New York, as United Nations delegates adopted the Sustainable Development Goals this week, celebration met a stern reminder: the goals cannot be achieved without a way to monitor and measure progress toward them. And with that call to action, big data companies are racing to position themselves as ideal partners for development organizations seeking better analysis in real time. Last week, Premise — a San Francisco-based company that collects economic data from a global network of contributors — might have pulled ahead in that race. Valor Equity Partners, building on a tradition of helping high-profile companies like Tesla and SpaceX scale, pitched in $35 million to a $50 million round of funding help Premise grow. “We believe that infrastructure for data collection is being remade through the application of modern technology. We have invested heavily in this theme and believe that Premise is at the forefront of the transformation,” Antonio Gracias, CEO of Valor Equity Partners, told Devex. “After a careful review of all the collection systems and data analysis platforms that we could find, we judged Premise to be unique in their approach and singularly valuable to their customers in the information they provide,” Gracias added. 2015 was a big year for Premise, which grew out of an email exchange between two alums of the tech world and has since expanded to 200 cities and towns spanning 32 countries. The company, which pays small fees to individual contributors for data points that can drive global decision-making, received funding from the Bill & Melinda Gates Foundation and partnered with the World Bank. Gracias joined the board of directors at Premise, which brings together established and emerging foreign aid leaders between its board and its advisors, including Rajiv Shah, the former head of U.S. Agency for International Development, and Hal Varian, the chief economist at Google. “We are a really awesome hybrid of technology and human intelligence. We’ve got people in all these places who tell us things we could never know. And they’re paid to do it, so the incentives are aligned,” David Soloff, Premise’s co-founder and CEO, told Devex. After the Philippines introduced a “sin tax” on alcohol and tobacco, compliance was tricky to track. Premise contributors uses Android apps to report on which brands and shops carried the required tariff stamps on these products. “This I think represents a template for how the international development organizations, a technology company, and the national [statistics] offices can all work together,” Soloff said. He spoke to Devex from an office the team is quickly outgrowing, and from which it has plans to relocate soon, describing models of partnership that could help traditional donors hand off tech-enabled data projects to government ministries. “So the World Bank introduces tech, they fund it for the first year or two, they turn the Ministry of Finance from totally suspicious to like, ‘holy cow, this can actually work,’ and now there is the possibility of contracting with the government ... after the World Bank leaves.” On the sidelines of the U.N. General Assembly this week, the potential for those kinds of partnerships took another step forward with the formal launch of the Global Partnership for Sustainable Development Data. The partnership aims to connect data collectors with development policymakers — and to equip those policymakers with the training and skills to use data for decision-making. Kai Kaiser, a senior economist for the World Bank, first connected with Premise at a World Bank event on big data innovation in Washington, D.C., and within a few weeks they began a formal partnership. He explained that current methods to measure this kind of tax implementation are expensive, paper intensive, and lack the geographical coverage to give a sense of what is happening at the granular level. But Premise promises to change that. The project in the Philippines delivered more than half of its observations from rural convenience stores other methods failed to track. This allowed the government to target pockets of noncompliance and remind them of the new regulations. “I'm starting to get the sense that there's a readiness for an upgrade in data gathering and analytics infrastructure across the board,” Kaiser said, adding that the international development sector might be witnessing, “the nascent days of an upgrade cycle.” “At the World Bank, we're thinking heavily on how emergent technologies can improve outcomes and speed up how we make decisions,” he said. “What's heartening about these solutions coming from Premise is how quickly they can move and get things done. It means we can make high leverage decisions faster and more efficiently — and ultimately the hope is that we yield much better outcomes than we have in the past.” Members of the 25,000-person contributor network, which Premise reaches via ads or links on WhatsApp and Facebook, are paid, ranging from 6-40 cents depending on how difficult their observations are to obtain. Premise then indexes and analyzes this data, making connections that can help drive decisions. In one example of this process, Premise used onion prices to point to inflation in India months before the government was able to do so. “In order to make sensible economic decisions it is important to have timely data that you can trust,” Varian told Devex in an email exchange. “Premise can provide customized, rapid, authenticated data from around the world in countries at every stage of economic development.” Varian added that real-time data collection is still in its early stages, with nontraditional data sources playing a growing role in the developing world. Soloff explained to Devex that the next step is for nontraditional data sources like mobile apps, drones and satellites to collaborate rather than compete in order to add value for clients like the World Bank. “Premise is mastering the last five feet of data collection and building a data analysis platform to extract valuable analytics from this information. There’s been early and encouraging validation from clients like the World Bank and the U.N. — and this is all the more important right now while all eyes are on the SDGs,” Gracias said. “But in my estimation, Premise has only begun to scratch the surface in terms of the massive impact it can have on global development.” Soloff said part of his job as CEO is packaging what Premise can offer the global development community in a very concrete way. He held up his hand and tapped his fingers as he spelled out: government revenue recovery, resource security, geospatial monitoring and infrastructure security. “It’s our belief that nothing should be earmarked unless it’s got a clear set of measurement criteria assigned to it,” he said. “It’s an absolutely monstrously huge market.” With $50 million in new investment, Premise plans to invest in efficiency gains in its information supply chain. The company is taking a step closer to the forefront of a data revolution that better get underway soon if SDG enthusiasts are going to meet the expectations they set this week. Read more international development news online, and subscribe to The Development Newswire to receive the latest from the world’s leading donors and decision-makers — emailed to you FREE every business day.