Is big data really the “next big thing”?
Google Trends, which measures the popularity of search terms, shows a continued explosion in the graph for “big data” searches in the past three years. The term has increasingly cropped up in discussions between development professionals as well.
But what is it, really? And how can it help identify and solve problems in the global development industry?
“I found 42 different definitions,” said Leopoldo Villegas, an epidemiologist who serves as a senior infectious diseases specialist at ICF International, a U.S.-based technology solutions company. “They’re driven by who is using the data, which at first has mostly been the private sector.”
Despite the cacophony of terminologies, there appear to be some areas of consensus. Big data is clearly a result of the digital revolution, which allows organizations and even individuals to collect and store more information than they could possibly process using conventional methods.
After going through all 42 definitions, Villegas came up with one that seems to work: Big data represents “data sets that are generated by a variety of instruments. They are large, diverse, complex and longitudinal [providing new information over time]. The last is the most important.”
Processing big data involves a sophisticated analysis of those data sets, which generally requires the services of specially trained data scientists. But what really matters is what you do with the data, noted Arijit Sengupta, CEO of BeyondCore, an automated business data analysis firm in Silicon Valley.
See more stories on big data:
● Big data for development: Insider tips from the experts
● Open data and privacy not 'headbanging' concepts
● Data revolution: It's not just about technology
● A bumpy road: Using data to inform policymaking
● The next steps for a 'data revolution'
● How the data revolution could transform development (but might not)
“In reality, it’s not about the size of your data, it’s about how you use data to create value,” he said. “We propose starting from what is actionable and then using automated analysis to evaluate millions of patterns to find the few cases where changing an actionable variable is likely to cause the greatest impact. To do this automatically, you need to evaluate large volumes of data at the finest level of detail possible — in other words, big data. But the data is big not because of its size, but instead because of its potential impact.”
The assumption is that these large and diverse data sets can be used to reveal solutions that might have been overlooked in the old days.
“You can answer the questions you want to answer,” Villegas said, “such as where to put a new family planning clinic.”
Proper data analysis can help organizations save money, which is not inconsequential in these days of flat or falling budgets, he noted.
“You can know [which clinics] are out of vaccines and which are out of HIV drugs,” Villegas said. “Now you send one truck out and then another truck and then a car” with individual deliveries. With better data analysis, supplies can be coordinated in a single dispatch, for instance. “You get better service delivery.”
To be effective, the data must be processed, analyzed and presented in an understandable and useful way within a time frame that allows actors to respond promptly and adequately, to paraphrase part of a definition offered by Global Pulse, a U.N. initiative that is examining ways to use big data. Villegas points out, however, that the time frame can vary according to the phenomenon in question. For malnutrition, an adequate response time might be measured in months; for starvation, in weeks; for cholera, in days; and for an earthquake, in hours.
Equally flooded with data but generally less well-endowed financially than their private sector counterparts, development and humanitarian aid organizations are beginning to play catch-up in the big data game. Yet so far they have precious little feedback about what works and what doesn’t.
“We need to document the best practices,” said Nuria Oliver, scientific director at Telefónica, the Spanish telecommunications firm, a ready partner in several development initiatives. “This is all novel.”
While hardly exhaustive, Devex has gathered eight examples of how big data is being used effectively by the global development community.
1. The U.S. Agency for International Development’s Demographic and Health Surveys program.
The DHS program has collected, analyzed and disseminated data on population, health, HIV and nutrition through more than 300 surveys in at least 90 countries. In the Democratic Republic of the Congo, for example, DHS has helped officials channel technical assistance from the national level to local health zones — areas that are roughly equivalent to counties in the U.S.
“Now they are moving to health facilities and communities,” Villegas said. In northeastern Brazil, the city of Natal used DHS data in an effort to tackle infant mortality as part of its drive to meet the Millennium Development Goals.
2. Nigerian health centers.
Nigerian authorities have mapped the country’s local health centers. This allows for what Villegas called “stratification in epidemiology.” In practice, this facilitates customization to address the special needs of specific areas.
“[Y]ou don’t do everything the same way everywhere,” he said.
3. Camfed and the education of girls in Africa.
Based in the United Kingdom, Camfed fights poverty, gender inequality, and HIV and AIDS in Africa by investing in girls in rural Africa. With programs in Ghana, Malawi, Tanzania, Zambia and Zimbabwe, Camfed provides grants to poor families if they keep their girls in school.
Using technology provided by Washington, D.C.-based startup Magpi, a provider of mobile data collection apps, Camfed has been able to accelerate and improve the collection, analysis and delivery of school attendance figures.
4. CGIAR hackathon.
CGIAR is a consortium of international agricultural research centers. During the 20th U.N. Climate Change Conference in Lima, Peru, in December 2014, it threw a hackathon to commemorate the launch of a program to provide public access to its data, the Amazon Web Services cloud.
The 24-hour event attracted software developers and computer programmers from across Latin America and the Caribbean. The first prize went to a Colombian team called Geomelodicos for a solution that can help farmers more accurately predict when to plant their crops each season now that climate change has rendered their old methods unreliable. A Peruvian team called Viasoluciones tackled water scarcity with a solution designed to help farmers make better decisions about how much water to use for irrigating different crops.
5. Digital Green.
Digital Green is a nonprofit that undertakes agricultural extension work in India, Ethiopia and Ghana. In India, for example, the organization collects data about its instructional video sessions in some 2,500 villages. A mediator encourages farmers to ask questions and engage in discussions, also collecting data about attendance, intentions to try new techniques and information on the specific questions that farmers typically ask.
The data are used for oversight and feedback.
“It looks like we’re getting a huge amount of data for free,” said Kentaro Toyama, associate professor at the University of Michigan’s School of Information and chairman of the Digital Green board. “But it takes a huge amount of human effort to collect.”
6. Flooding in Mexico.
Telefónica worked with the Mexican government and the World Food Program to see whether activity in its cellular towers could help provide information about where flooding is taking place during major storms.
“We did find that there were correlations,” Oliver said. “This could help the government understand where its relief efforts should be focused.”
7. Mexico’s response to the H1N1 flu outbreak.
Telefónica and the Mexican government also joined forces to analyze the effectiveness of the country’s energetic reaction to the swine flu outbreak in 2009. Airports and businesses were closed, and much of the country ground to a halt for several days.
“Was it worth it?” Oliver asked, repeating the question posed by the Mexican authorities.
The telecommunications company developed a couple of alternative scenarios to help figure out whether reduced mobility contributed to the fight against the disease.
8. Orange’s Data Challenge for Development.
In 2013, through its D4D challenge, the French telecommunications company Orange invited researchers to analyze its mobile network statistics in the Ivory Coast in an effort to contribute to the West African country’s socio-economic development. Prizes were awarded for the best solutions.
Outcomes included solutions designed to predict the outbreaks of epidemics, improve crisis response, address drought-related problems, improve infrastructure and develop new public services. A similar campaign kicked-off in Senegal last year and runs through April 2015.
Know other examples of how big data is being used effectively by the global development community? Let us know by leaving a comment below.
Read more international development news online, and subscribe to The Development Newswire to receive the latest from the world’s leading donors and decision-makers — emailed to you FREE every business day.
