Opinion: Digitalization helps health systems, but we need more evidence

A ministry of health HIV prevention program manager faces a critical decision: should she invest the program’s limited resources in a new, more effective drug, or in a new digital system that could expand the reach of the existing therapy to prevent HIV infections? How does she decide?

Digital health interventions, which encompass all the technological systems, software, services, and resources that contribute to the digitalization of health data and health care delivery systems, have generated well-deserved hype in the global health community for well over a decade. Digitalization is widely recognized as increasingly necessary to modernize health systems. As a result, governments and donors are making significant investments in digital health interventions and the required underlying digital public infrastructure to meet a range of needs — improving the quality and coverage of patient care, decreasing the workload of health care workers, and strengthening health systems.

But as digital health matures and becomes more widely implemented, a stubborn gap remains: evidence. Which interventions are working? Which are cost-effective?

Because digitalizing health systems has the potential to improve access to health care, provide better-quality data for decision-making, and save health workers’ time, we expect it to result in more cost-efficient systems and better health outcomes. But we need more quantitative evidence to back up those claims and convince decision-makers — such as the HIV program manager above — to invest. Additionally, decision-makers need to be able to compare investment options before making a choice.

Unfortunately, quantifying the costs, impact, or cost-effectiveness of digital health interventions is a challenge, largely due to a lack of agreed-upon methodologies and insufficient funding for evaluations. Too few studies exist, especially in low- and middle-income countries, that provide value for decision-makers.

A recent supplement highlights a handful of these studies, illustrating the feasibility of such research. But to sustain digital health investments, more impact, and cost evidence is sorely needed.

Digital health interventions can save time, money, and lives

When assessing the value of a vaccine, drug, or test, it’s relatively straightforward to determine that for every X dollars we spend on this vaccine, we will save Y lives or Z hospital visits. In other words, we can easily calculate the cost-effectiveness ratio of vaccines. This is much more difficult to do for digital health interventions, which often have indirect, complex, or wide-ranging benefits for health systems.

Taking this complexity into account, PATH’s Digital Square initiative has developed a Total Cost of Ownership Tool to help stakeholders assess digital health intervention costs.

To encourage more evidence generation and share these methods, Digital Square led the development of a supplement in Oxford Open Digital Health focused on cost and impact evaluations for digital health interventions. The results show not only that is it possible to conduct such evaluations on a wide range of intervention types, but also that digital health is, most of the time, both impactful and cost-effective:

• An impact evaluation in Kenya estimated that even a small 5% increase in treatment coverage attributable to iHRIS — a digital human resource management tool for health care — could reduce severe malaria cases by 15% and deaths by 13%. iHRIS is also a global good, a digital health tool that meets the criteria for open-sourcing, interoperability, scale, and effectiveness.

•  In Lilongwe, Malawi, an analysis of health care workers’ time spent on data collection and entry estimated that the introduction of the Community-based ART REtention and Suppression, or CARES, app would reduce data-related time by 60%.

• Also in Malawi, a cost-effectiveness study of two-way texting, or 2wT, intervention — which sends visit reminders, motivational messages, and supports direct communication between patients and providers — on retention for antiretroviral therapy estimated that, on a limited scale, 2wT was costlier but more effective than the previous standard. However, expanding access to 2wT could result in cost savings.

• In Zambia, a cost-effectiveness study of a digital geospatial tool called Reveal — a global good added to a 2017 malaria control campaign for indoor residual spraying, or IRS — estimated that using the tool with IRS is cost-effective in the Luapula province.

Other research in the special collection found that a mobile learning platform for community health volunteers increased health care facility referrals in Kenya and a digital clinical decision support tool for health workers increased clinical guideline adherence in Somalia. One evaluation in Nigeria showed a lack of overall impact on the main outcome of interest (business performance and client satisfaction), largely due to a lack of user engagement with the digital platform.

Conducting more studies such as these will help the digital health community learn from what does not work and increase investment in what does work. This evidence adds value for countries and the entire digital health ecosystem.

Donors and governments need more evidence

Without evidence of the cost, impact, and value of digital and data technologies for health, digital health stakeholders have a limited basis from which to incentivize, justify, and allocate funding. This, in turn, impacts the sustainability of digital health programs.

In interviews, country government stakeholders reinforced the need for more data on the costs and return on investment of digital health interventions. Donors and funders emphasized the need for impact evidence to show that their investments are achieving their intended results — improving health, saving lives, and increasing system efficiencies in costs, time, and human resources.

There is a large, unmet demand for a better understanding of the impact, cost, and value of digital public infrastructure and digital health investments. Evaluating impact can help decision-makers hold developers and implementers accountable and improve the allocation of limited resources.

By funding, demanding, and standardizing digital health evaluation methods, we can all help make the digital transformation of health systems a sustainable reality.

Learn more: Oxford Open Digital Health Supplement: Health Impact, Efficiency Gains, Costs, and Savings from Digital Health Interventions — Digital Square