Big data for development: Insider tips from the experts

As it has evolved in the private sector, big data analysis has become a high-end affair involving sophisticated operations performed by a new set of highly trained specialists. There’s a shortage of those specialists — and they’re coming at a premium. “Demand for computer systems analysts with big data expertise increased 89.9 percent in the last 12 months,” said Louis Columbus, a Forbes.com contributor, in a recent post. “The median salary for professionals with big data expertise is $103,000 a year.” If companies are scrambling to hire big data specialists with even minimal qualifications, where does that leave low-budget development organizations and implementers? Devex spoke to several big data experts for advice on how actors from the development cooperation and humanitarian aid sphere can effectively use big data to their advantage. Here are their 12 top tips: 1. Define your term. Most experts agree that not everyone means the same thing when they use the term big data. Leopoldo Villegas, an epidemiologist who serves as senior infectious diseases specialist at ICF International, a U.S.-based technology solutions company, counted 42 different definitions. No matter. Just make sure that you and your partners agree on what you mean. “Big data is any data that is useful to you,” said Arijit Sengupta, CEO of BeyondCore, an automated business data analysis firm in Silicon Valley. “Then you look at it at the finest level possible.” 2. Don’t get bogged down by data. A corollary of the first tip, this rolled off almost everyone’s tongue. “It is wrong to focus on the volume,” Sengupta said. “That’s like bragging about how much stuff you have in your attic.” As Nuria Oliver, scientific director at Telefónica, the Spanish telecommunications firm, put it: “Data is digital garbage. Some people focus on having data, but so what?” Instead, she said, try to make sure that you have the right data to answer the right questions. 3. Include grass-roots input. Not just in data collection, but also in determining the questions. In this more data-centric environment, Oliver noted that leaders should ask people on ground: What do you need to know to help you get things done better? There is also a need to ensure you have access on the ground to a steady flow of good quality data. “You need to check them for consistency,” noted Villegas. One extreme example on his radar: Parts of the Democratic Republic of the Congo have sometimes gone months without producing any data, for instance, when roads have been destroyed in the fighting that has plagued that country. 4. Concentrate on the results. “If you are not careful, you’ll end up the same as a lot of Fortune 500 companies [which] are doing analysis for the sake of analysis,” Sengupta noted. “You need to focus on outcomes and not on the mechanics.” Avoid this classic “hammer looking for a nail” dilemma, warned Kentaro Toyama, associate professor at the University of Michigan’s School of Information and chairman of the board of Digital Green, a nonprofit engaged in agricultural extension work in India, Ethiopia and Ghana. 5. Try ready-made solutions. One solution has been developed specifically for development and humanitarian organizations. Based in Washington, D.C., Magpi was founded in 2003. The firm evolved from the experiences of founders Joel Selanikio and Rose Donna, who “pooled decades of experience in global health, disaster response, technology and international development to develop technology that would enable easier mobile data collection for NGOs and nonprofits.” Selanikio recalled his work as a consultant training people on how to compile data using electronic devices, instead of paper forms — which then had to be keyed into computers. Magpi offers a seamless fieldwork-to-dashboard digital solution that has been adopted by Camfed, John Snow, Inc., the Kenyan Ministry of Health — working with UNICEF, among others. “We automated ourselves out of the picture,” Selanikio said. “We were the ones who would have to fly out to train people.” 6. Forge partnerships with technology startups. Talk to technologists. What’s in it for them? “They need to prove [the effectiveness] of their technologies,” Sengupta said. “They get to make sure their technology works, and get a great case study.” Sengupta said that even his firm, which is too well-established to be called a startup anymore, welcomes proposals from nonprofits. “We are always happy to do analysis for free for true nonprofit work,” he said. 7. Consider making your data freely available — and let analysts have a go. Both startups and large corporations need to test their products, Sengupta noted. “Good data is hard to come by,” he added. Your data could just make their day. And their conclusions might make yours. CGIAR, a consortium of international agricultural research centers, has chosen this path. It is making its data freely accessible via the Amazon Web Services Cloud for researchers who want to use it to address food security and development challenges. 8. Build capacity. “If a big data project is going to do any good, you need qualified people,” Toyama said. “I hate to say ‘data scientists,’ but you need people who are trained and prepared to interpret the data.” Policymakers and decision-makers need good advisers who understand big data, he added. This reinforces calls for improvements in education. “You have to build capacities at the country level,” Villegas said. “To create that culture will take years — to instill the idea of systems thinking, that things are connected.” 9. Don’t forget ethical and privacy issues. This might seem obvious, but deserves notice. “We have a clear code of conduct,” Telefónica’s Oliver said. “It has to exist — even if the data is anonymous.” “In some countries,” Villegas noted, “you cannot say that some areas have more cases of HIV than others.” 10. Be aware of false positives. “You can always slice-and-dice to see a beautiful pattern,” Sengupta noted. “That does not [necessarily] mean that it is actionable. If the variability is high, it could just be an accident.” Just because there is a correlation between drowning and ice cream sales does not mean that one causes the other, as a U.N. Global Pulse document noted. Common sense tells us that people are more likely to drown on hot summer days at swimming pools and beaches — precisely when and where ice cream is likely to be popular. “There are trends that are not trends,” Toyama said. 11. Remember that you operate in the real world. “Big data has three challenges in international development, none of which are about big data per se, but rather about human institutions,” Toyama said. “First, you need excellent data collection, which is a challenging task in developing communities. Second, data always requires interpretation — all the data in the world is of no use if you don’t have people who understand the limitations of the data and can then come to the right decisions. Third, you need a strong institution on the ground to implement decisions.” 12. Don’t expect too much. “Big data is big, but it is not going to work alone,” Villegas said. Oliver meanwhile concluded that actors “have to understand the values, but also the limitations. It is a partial view of reality. It is important to understand the potential limitations and biases.” Are you working with big data? How can development cooperation and humanitarian aid actors harness big data to their advantage? Share your thoughts by making a comment below. Check out more insights and analysis provided to hundreds of Executive Members worldwide, and subscribe to the Development Insider to receive the latest news, trends and policies that influence your organization.