Opinion: Are health systems really measuring what matters?

In the realm of health care, one of the most pervasive metrics used to evaluate “success” has been the simple counts of events. From the number of people treated to the number of surgeries performed, it’s common to see numerical tallies that judge “performance.”

While counts may provide valuable insights into workload and capacity, they are limited in their ability to measure improvements in health outcomes, the effectiveness of interventions, or the health care system as a whole. Sometimes they can be effective as a proxy, such as the number of children vaccinated as the outcome of vaccination is usually known. Other times the impact can be far less certain — for example, the number of people screened. What happened next? How many were diagnosed? Were they then treated? If treated and diagnosed, how many improved?

Counts are convenient. Yes, they offer some insight into the volume of activity within a health care system. But they do little to explain the quality or impact of a particular action. Focusing solely on event counts promotes a “quantity over quality” approach — and the problem with that is it overlooks the broader goals of health care: optimizing health, enhancing care delivery, reducing inefficiencies, and improving effectiveness.

Counts don’t consider bottlenecks in the system — the time to return pathology results, for example — nor their impact on treatment outcomes. Systemwide measurements are very difficult to develop and report. As a result, measures of convenience often trump the development of more complex — but better — systems of measurement.

The case for process improvement metrics

Evaluating health care effectiveness is about shifting focus from simple event counts to metrics that reflect process improvements and assess how care is delivered and how better care can be achieved.

A key element of process-focused measurement is adopting evidence-based best practices. For example, evaluating the implementation of standardized care protocols for breast or childhood cancers can reveal whether providers are adhering to guidelines that improve health outcomes, including the provision of relevant supportive care actions that ensure treatment effectiveness.

Of course, there are challenges, particularly as process improvement metrics require a more detailed investigation of baseline metrics as well as measures that demonstrate progress — that includes supplementary measures for identifying rate-limiting steps along the way.

Take a moment to consider where “measurement and evaluation” often sit in budget proposals for new projects. In many cases, it’s in the second-to-last line item, right before “dissemination and publication.” Subliminally, this frames measurement and evaluation as a final step, rather than an integral part of the planning process. To truly create an effective measurement platform, ensuring that it is close to the top of a budget proposal will allow for the appropriate commitment to clearly planning metrics from the outset.

This was a key lesson for us at Amgen. After creating a new team focused on health systems strengthening and access to health care in low- and middle-income countries, we developed a measurement and evaluation framework that would consider the breadth of possible measurements — from simple to complex. We were keen to move away from simple measures and instead focus on ones with a more person-centered and systems-centered impact. Crucially, we wanted to commit to the principle that outcomes are more important to counts. Thus, developing a framework for how we consider measurement and evaluation from the outset was something to guide us and our collaborators into making informed measurement choices.

Focusing on outcomes

When it comes to complex, multiyear programs, process metrics are a crucial interim milestone. That said, they must be linked to the ultimate goal of health care: improving health outcomes.

This leads to an interesting budgetary question: Should we increase the allocated budget for a more complete understanding of process and outcomes? Should we reduce the number of people served in order to do so? The answer is probably yes, particularly if we believe in the “quality over quantity” ethos.

This brings me back to my earlier argument: Measurement and evaluation should take a more prominent place in the budget creation process. It also relies on consultation with the system implementer or system beneficiary — what outcomes are important to them? Clearly, qualitative metrics, such as improvements in a person’s quality of life, will be important to recipients of the intervention and being cognizant of that is a critical component of person-centered care.

Outcome-based metrics are the gold standard for assessing health care system performance because they directly address the effectiveness of care delivery. However, while some outcomes can be counted — such as return to mobility following hip fracture surgery or replacement due to minimal trauma fall — not all that can be counted are a good outcome measure. For example, was the fracture due to osteoporosis and did the person subsequently start high-quality osteoporosis treatment? Did their bone density increase over 12 months?

Similarly, tracking patient-reported outcomes allows health care systems to measure how treatment affects the person’s overall health and quality of life, rather than focusing on clinical measures like blood pressure or cholesterol levels.

Implementing a comprehensive framework

So, what does it take to successfully shift toward a measurement and evaluation framework that emphasizes process improvements and outcomes? Thoughtful planning and coordination.

That means investing in data-collection infrastructure that can track both process and outcome metrics in real time. Electronic health records and patient registries are critical tools for capturing necessary data and making it accessible for evaluation. However, to be impactful, that data must be used for measurement purposes.

We must also consider that measuring the impact of health interventions will become much richer with the adoption of technological innovations like machine learning and artificial intelligence. Leaving a legacy of effective measurement and evaluation following a program and building a base for future predictive analytics — even in low-resource settings — will better equip a “future-ready” workforce.

Driving change in health care systems

Yes, a comprehensive measurement and evaluation framework focused on process improvements and outcomes is complicated. But it’s by no means impossible.

Does it decrease the potential impact of an immediate program by trading program budget for better planning? Possibly — but only if we measure ourselves by the task at hand, rather than the potential to be more relevant and influential in the longer term. We have to be rigorous about measuring what matters.

By turning our focus to the quality of care, the actual improvement of relevant and person-centered health outcomes, and measures that assess efficiency and effectiveness of health system changes, we feel that our contributions are better set to deliver more efficient, equitable, and impactful care.

This is an opportunity to challenge ourselves as a health care community — to move beyond mere numbers and start measuring what truly matters: The health and well-being of the people and communities we serve.