Opinion: The coming evaluation revolution in corporate responsibility

An employee checks a shipment headed for Mozambique for the US Presidential Malaria Initiative at the worldwide distribution center for the Novartis antimalarial in Basel, Switzerland. Photo by: Mark Tuschman / Novartis AG

If intentions were enough, the world would be a much better place. Most companies want their corporate responsibility programs to work, but few develop the evidence base to understand whether they actually do.

There's a pervasive belief in the good deed: That well-meaning investment will automatically produce positive results. That’s wrong. It's assumptive, not based on scientific insight, and means we pass up the opportunities to learn how to do better.

Too often, we measure outputs — pills distributed, lavatories constructed — without looking at whether people in the targeted area actually have more access to medicines or safe water as a result. We measure these things because they are easy to count. However, they can also be misleading: Who is taking the pills? Who is using the lavatories? Has better access to care or sanitation actually improved anyone’s health?

We should be measuring the impact on variables that we value as a society: Has a program improved health? Has it created new economic opportunities for women? Has it reduced environmental damage? Often, though, these hard outcomes seem too daunting to capture.

Ripple Effect: The expansive impact of NCDs

While combating noncommunicable diseases is a global task, regional contexts cause unique challenges. In this visual series, we explore a few of these challenges through the stories of health care workers and individuals living with NCDs in three different countries.

The more frequently companies can link social interventions and business models with their core strategic priorities, the more important it will be they understand the impact of these programs. Such measurements enable us to improve program management and resource allocation. Understanding the performance of an investment is not just good management practice: We need to understand exactly how social impacts and improved business performance fit together. And this will be how we can finally achieve social change at scale.

Measuring outcomes, not just outputs

For the past two years, Boston University researchers have been evaluating the impact of Novartis Access, a program against key chronic conditions, on Kenyan households where a resident has one of four specified noncommunicable diseases.

The Boston University baseline study on Novartis Access was conducted in eight counties in Kenya in 2016, asking patients diagnosed with 1 of the 4 chronic diseases whether they had their prescribed medicines available in their home. They also asked how much of their household health expenditure was spent on these medicines.

Researchers found that, as in several other African countries, awareness about NCDs remains low, with a lack of specialized doctors and medical training, despite a high mortality rate in Kenya due to NCDs.

Only 60 percent of people who had been prescribed a medicine had it in their home. And the amount they spent on treatment was high — some 75 percent of their overall household health spending.

More on Novartis Access

Novartis Access offers governments, NGOs, and institutional customers in lower-income countries a basket of 15 medicines targeting the most common NCDs at a price of $1 per treatment per month to public sector purchasers. All the medicines are among the most commonly prescribed ones in their therapeutic area, they have been qualified for use in tropical climates, and they target cardiovascular diseases, type 2 diabetes, respiratory illnesses, and breast cancer. In order to strengthen health care systems, the program also partners with governments and NGOs on capacity-building activities to improve the management of chronic conditions. Read more.

They also found that while more than 50 percent of chronic diseases are diagnosed in the public sector, more than 40 percent of patients actually buy their medicines in the private, for-profit sector.

Let’s take the case of asthma: Half of those studied who knew they had asthma did not have any asthma medication at home. Of those who did have medicine, many had tablets, not the inhalers recommended by international treatment guidelines. Most surprising of all was that among patients who had a salbutamol asthma inhaler in their home, those in the poorest quintile were least likely to have received the medicine free of charge.

For each medicine researchers studied, it was some of the poorest patients who paid systematically more than wealthier patients; the poorest often come out worst even in accessing subsidized treatment.

The researchers returned for a mid-point evaluation in 2017, using a cluster-randomized trial design, and are conducting an end-point evaluation this year. This final evaluation will measure whether Novartis Access has improved the health status of patients with NCDs, not just whether it has improved access to diagnosis and affordable treatment. This detailed impact analysis will allow those running Novartis Access to adjust the program as it is rolled out in further counties.

This is, it seems to us, a sound example of academics and a pharmaceutical company collaborating to test whether a corporate responsibility project is having the impact it sets out to make. We now want to encourage other companies to be more systematic with their monitoring and evaluation as well.

Here are two ways this can be achieved:

1. New relationships between universities and the private sector

If companies provide open-source data on their social innovations to the public, they create a social good. They show other companies where to invest in future and they show society which corporate responsibility initiatives to nurture and encourage. These data must be independent and reliable, and it is here that academic institutions can play a central role.

Universities can be a neutral space to be a trusted anchor, with the legitimacy to act as guardians for such a database of open-source data. It is the role of academics to ensure that access to these data is offered transparently and in a form that can easily be used by other researchers.

There is a huge variance in academic institutions, and it will not necessarily prove a fit for them all. It will be important to think through all the long-term commitments involved for all parties.

2. Frameworks as tools for accountability and learning

Measurement and evaluation frameworks can also be a learning tool. As stakeholders with different priorities come and go, this means the frameworks need to be more agile. Social interventions and inclusive business models often operate in unstable health systems, so the exact conditions are not fully known at the point when an intervention or business model is designed. As a result, they need to be able to respond dynamically to different or changing conditions.

Measurement and evaluation provide the information and insights to enable these programs to adapt and respond in a fast and meaningful manner. But this also requires that the measurement frameworks are open to adaptation and respond to changes with a flexible but still rigorous design.  

When evaluation frameworks are robust, they force stakeholders to articulate and develop a shared language, and also a shared understanding. There needs to be disagreement, even conflict, to come to something that is really meaningful, instead of superficially agreeing on platitudes. This can be a bumpy ride, but a journey worth taking.

Evaluation needs to change

It is time for an evaluation revolution. This requires all of us, in all our different roles, stopping to think through the frameworks we use to monitor and evaluate interventions.

Painful as it is, we need to accept that our intentions are not good enough. That good deeds sometimes have unintended consequences, or even fail to change anything at all. It is better to know and change, rather than hope for the best and push forward blindly.

For more coverage of NCDs, visit the Taking the Pulse series here.

The views in this opinion piece do not necessarily reflect Devex's editorial views.

About the authors

  • Harald Nusser

    Dr. Harald Nusser has been leading Novartis Access since June 2015. Since October 2016, he has also taken on the responsibility for the newly formed unit, Novartis Social Business. Dr. Nusser started his career at Schering AG in 2000 as a mathematician in the global research and development function. Having worked in both exploratory research as well as clinical research, he held positions of increasing responsibility between 2004 and 2010 in Corporate Strategy and Strategic Planning in both Schering AG and as of 2006 in Bayer in various senior positions. He is also a member of the Novartis Access to Medicines Committee.
  • Johanna Mair

    Johanna Mair is professor for organization, strategy, and leadership at the Hertie School of Governance in Berlin. Her research focuses on how novel organizational and institutional arrangements can generate economic and social development. Mair also is a distinguished fellow at the Stanford Center on Philanthropy and Civil Society where she serves as a co-director of the Global Innovation for Impact Lab and as the academic editor of the Stanford Social Innovation Review.
  • Michael Fürst

    Michael Fürst has 20 years of experience in corporate responsibility, social business and innovation, and integrity and compliance management, in business and academia. He is currently having responsibilities for the implementation of the Novartis CR and access to medicines strategy, inclusive business model innovation, innovative finance, ESG investor relations, measurement and evaluation, materiality and stakeholder engagement.