How Indian development data is disappearing

India began its six-week election process last week, with incumbent Prime Minister Narendra Modi vying for a third term in power. Modi has touted impressive figures on economic growth and poverty reduction during his 10 years as prime minister. But former government officials have said these claims are flawed, to the extent that some have stepped down over what they say is the misuse of data to support a political narrative. During its G20 presidency, India promoted a policy of data collection and analysis as key to achieving the Sustainable Development Goals, calling for the sharing of high-quality granular data to inform policymaking and improve public service delivery. But India’s own statistical systems do not appear to reflect this ambition. Data in many key areas of development has been conspicuously lacking in recent years — leaving policymakers and nonprofits relying on data that in some cases is over a decade old. The most glaring example of this is the census, which has not been conducted since 2011. While it was initially postponed due to the COVID-19 pandemic, the reason for the ongoing delay is unclear, said P.C. Mohanan, former chairman of India’s National Statistical Commission. “None of us can answer why it is not happening, we just know that we know it's not there,” he said. “So we are left with using the 2011 population data for most of the target setting or adopting policies.” This also has a knock-on effect for national and state-level surveys on topics such as health and employment, which rely on the census data to provide a sampling frame. Given India’s rapid population growth and migration in the past decade, this leaves large numbers of poor people in India uncounted, said Mohanan — and potentially excluded from public services that they would otherwise be entitled to, such as essential food subsidies. “The entire principle of doing development at scale actually rests on the availability of accurate, authentic, and diverse data,” said the CEO of one leading nonprofit working in India, who spoke to Devex on condition of anonymity. The NGO in question relies on data from national and state-level surveys on development to target their interventions and understand the evolving needs of the communities it works with. “If the country is moving towards wanting to be a developed country, deploying data strategically, then the majority of the country will be deprived of that journey for want of accurate data,” the leader said. Missing data In the years after independence, India built a comprehensive statistical system, employing innovative methodologies to capture survey data on consumption expenditure, employment, and human development indicators. But in recent years, there has been a “deep decay in the quality as well as the quantity of Indian statistical surveys,” said Pranab Bardhan, an Indian economist and professor emeritus at the University of California, Berkeley. Part of the problem is the huge logistical feat presented by a country the size of India, which overtook China last year to become the world’s most populous nation. The statistical system has struggled to keep up, Bardhan said. “As the economy became more complex, the data needs became more complex,” he explained. “While that happened, the government did not give enough resources to this; not the amount of resources that you would otherwise need for such a huge exercise.” But the problem is also political, he said. “If data results are unflattering, then there's a tendency to either suppress those data, or publicly deflate the quality of those data,” he said. In the lead-up to the last national election in 2019, the government was accused of withholding employment data, which had been vetted for release by the National Statistical Commission, a government-funded but autonomous body tasked with collecting much of India’s economic data. In response, an open letter from over 100 of the world’s leading economists expressed concerns over “political interference” in India’s statistical data estimates, asserting that economic data was a “public good.” “In a democratic society, everybody has a right to know what is happening. And that knowledge has to come from independently collected data.” --— P.C. Mohanan, former chairman, National Statistical Commission The data, which was leaked and subsequently released by the government after the election, showed that India’s unemployment rate had reached a 45-year high of 6.1% in 2017-2018. Unemployment is seen as a key concern among Indian voters. The decision to withhold the employment report led Mohanan, then chairman of the National Statistical Commission, to resign in protest. “Slowly, I had a feeling that this commission was not being allowed to discharge its responsibilities, so I thought it is better to leave,” he told Devex. The publication of other important datasets, such as the Consumption Expenditure Survey, which provides India’s poverty estimates, has also been delayed or withheld in recent years. “Today’s government has a certain image of India, and if the data doesn’t back it, or the data puts it in a different light, it makes the government look vulnerable. And it doesn’t want to look vulnerable,” said the nonprofit leader. Adversarial data Due to the largely informal nature of India’s economy and low rural access to services, Indian administrative data — the information collected when people interact with public services such as hospitals, schools, and government programs — often provides an incomplete picture. Only an estimated quarter of registered deaths in India provide a medically certified cause of death, for example. It is also inherently prone to reporting bias, said K.S. James, former director of the International Institute for Population Sciences, or IIPS, because officials are motivated to demonstrate good performance. The unrealistically high figures which often result make the government look good and provide officials little incentive to improve the quality of administrative data, he said. Economists historically relied instead on survey data to fill the gaps left by uncaptured administrative data, or to hold its claims to account. But survey data which contradicts preexisting administrative data, especially that which has been used to tout the achievements of the government, is increasingly viewed as adversarial, James said. For example, administrative data from the Swachh Bharat Mission, a sanitation scheme launched by the government, show that 100 million toilets have been built since 2014, covering 100% of the country — causing Modi to declare India open-defecation free. However, data from the 2019-2021 National Family Health Survey, or NFHS, contradicted this claim, reporting that only 83% of households have access to a toilet facility and 19% of households still practice open defecation. This is down from 39% from the 2015-2016 NFHS survey, which should be a cause for celebration, Bardhan said. But by declaring “100% victory in this campaign” the credibility of the administrative data is called into question, he said. Shortly after the NFHS data was released, James — who at the time was the director of the IIPS, which is responsible for conducting the survey — was suspended. He later offered his resignation and has since moved to the United States. “Anyone who conducts a survey is under serious pressure now in India,” he told Devex. India’s health ministry did not respond to requests for comment. With survey data increasingly suppressed or under dispute, political debates on development and economic growth are often underinformed, Mohanan said. “Independent scrutiny of this data, that is actually not happening,” he said. “The government is free to make claims, and the opposition is also free to refute them, because there is no data. “In a democratic society, everybody has a right to know what is happening. And that knowledge has to come from independently collected data.” Data for development The lack of independent data not only threatens to obscure ongoing development challenges, but may actively make them worse, experts warn. “Many of the important development programs, health programs, have been implemented because the NFHS has shown a trend,” James said. By designing interventions around data which does not reflect reality, "we will be doing a disservice to the nation," he said. Gaps in the data from national surveys also affect the ability of NGOs to fundraise. Donors generally want evidence to demonstrate the scale of the problem and the likely impact of any intervention. But for an NGO to collect its own data would be expensive, time-consuming, and would put the organization at risk of punitive measures from the government, according to the nonprofit leader. Their nonprofit has stopped openly publishing some of its impact and evaluation reports, in case they are viewed as acknowledging a problem the government considers itself to have solved. “If you are collecting data it’s almost like declaring a war on the state,” the leader said. “If your data is saying there’s a problem, then their feeling is you’re questioning the government’s ability to deliver.” In recent years, thousands of NGOs, including Oxfam and Amnesty International, have been stripped of their foreign funding permits by the government or had their accounts frozen — effectively forcing them to shut down operations in the country. “This government is clamping down on civil society,” Bardhan said. “One of the main reasons for that must be that civil society organizations come out with results which are not that flattering.” But, argues the nonprofit leader, “Data by itself is not guilty of anything. It’s agnostic.” They believe that the current approach of the government may prove to be shortsighted. “In getting its present optics right, the lack of data will put us back in the development journey by at least two generations,” they warned.