Comparing numbers is an effective point of reference for organizations, but benchmarking comes with its fair share of caveats and problems, and pay for performance is making data comparison even more complex.
Benchmarking has been prevalent in other industries since the 1970s but is a newer concept in healthcare. Comparing data against one's peers to assess organizational performance seems simple enough, but benchmarking can wreak havoc if overemphasized. Further, pay-for-performance reimbursement structures come with their own benchmarking woes, but some health systems are finding effective ways to get a full picture of where they are and how much they need to improve.
Identifying peers
For hospital executives and managers, one of the most difficult parts of benchmarking is simply trusting the data in front of them. Igor Belokrinitsky, partner with Strategy&, has seen this firsthand. "It's very hard to convince people that benchmarks are right," he says. "Every hospital is different, and benchmarks are not completely comparable depending on how they report data."
Concerns about apples-to-oranges data comparisons aren't new, but it has become even more difficult for hospitals to find benchmarking peers given the widespread financial pressures hitting all organizations. Even hospitals historically seen as exemplars are feeling more pinched. As a result, hospital executives may feel less-than-confident in knowing which organizations to benchmark themselves against.
As Mr. Belokrinitsky says: Every hospital has a cost problem. And those cost problems — how to best trim 10, 20 percent from the cost structure, for example — are not easily solved by benchmarking. Executives should use these numbers as a back-pocket tool to identify problem areas or parts of the organization that are seriously askew. As discussed below, executives have plenty of considerations to make when looking over benchmark data sets, especially in light of new payment models.
Benchmarking in a new pay paradigm
Pay-for-performance reimbursement brings its own benchmarking nuances. Hospitals participating in accountable care organizations must comb through more cost and quality data, especially those organizations with both commercial and Medicare ACO contracts. Medicare ACOs, for instance, must track their performance on 33 quality measures. Many commercial ACO contracts contain metrics in line with those of Medicare, but there is no standardization in the particulars of private ACO contracts.
For hospital executives and ACO leaders, half the battle comes down to finding the sweet spot: metrics that are both meaningful and controllable, says David Muhlestein, PhD, director of research at Salt Lake City-based Leavitt Partners. These are the data points that, if improved, will contribute to better patient outcomes. They are also easy for ACOs to influence. But finding measures that meet this two-part description is tricky.
Take the tobacco use measure, for example. While meaningfully indicative of the health of a population, hospitals have a limited amount of influence on whether somebody uses tobacco. Other measures, such as medication reconciliation, are closer to ACOs' jurisdiction. "There aren't a ton of measures with consensus that improving the measure makes the population better and are also easy for the ACO to control," says Dr. Muhlestein.
Cost data is meaningless without quality data, and vice versa
Atrius Health, a Newton, Mass.-based nonprofit alliance of six community-based medical groups and a home health care and hospice agency, is one of CMS' 22 remaining Pioneer ACOs. In its second-year results, Atrius came in $3.1 million under its Medicare cost benchmark with a 92.5 score for quality — the second highest of all the Pioneers. "I would say we put a lot of pressure on ourselves," says Emily Brower, executive director of Atrius Health's accountable care programs. "Every month we look at our metrics, and we're probably much harder on ourselves than the public would be."
How does the ACO make sense of its many metrics and avoid data fatigue? One tactic is monthly meetings, appropriately nicknamed ACO Day, in which ACO leaders walk staff through the different ways they monitor performance and zero in on a few metrics. (In September's meeting, staff focused on post-discharge medication reconciliations.) Atrius takes pains to identify all the work, people and processes it takes to move the needle on each metric.
"If there's something on which we're not performing as well as we'd like, we break it down to figure out who owns what piece of that and how they can contribute to a better result," says Ms. Brower. "One of the great things about this work is when you improve quality, costs also come down. There's a lot of energy and excitement around this work."
Sharing ACO data also requires transparency by management. Atrius Health designated a portion of its website for its ACO quality scores, something the system internally decided to make available to consumers even though CMS didn’t initially mandate it. "We very much support the conversation around value, and to understand value, you need both cost and quality data," says Ms. Brower. "Cost data without quality is not the whole picture."
Hospital haircuts
With so much data out there, sometimes an organizationwide data dashboard is straightforward enough to engender confident decision-making among hospital executives. Black and white numbers let them see it is possible to operate a hospital at a certain cost level, adding conviction to the targets they set for managers.
But indiscriminate cost targets, or "haircuts," which are often set in an attempt to meet a predetermined benchmark, rarely result in true savings, says Mr. Belokrinitsky. The work to be done doesn't go away, but is instead episodically shifted elsewhere in the organization, which can hurt performance in the end. By basing budgeting or staffing decisions on benchmark numbers, leaders may "fix" a problem but never uncloak the cause. Why is physician productivity lower than that of your peers? Why is the operating utilization rate lower than your competitors'? Benchmarks don't tell those stories.
Benchmark-driven decisions can also make things immediately worse. After benchmarking, one health system eliminated some transporters to better align with its peers. Soon after, the system did not have enough transporters to take patients to the operating room and back. This left nurses transporting patients and delayed start times in the OR. Repercussions spread throughout the system, creating new costs that were significant compared to immediate savings from reducing the number of transports.
In another instance, one health system decided to reduce its workforce as a cost-cutting measure. The system didn't want to touch its fairly large physician base for political reasons, so instead, it laid off several nurses. The unintended consequences of this decision were soon evident. Physicians grew frustrated when they were tasked with taking vitals and bringing patients to rooms in nurses' absence. Their productivity dropped as a result of these added tasks, and patient access decreased as a result.
These unintended consequences suggest decisions should be based on more information than benchmarks can always provide. "When you look at staffing, you might have a number that says you're right on benchmark. But when you double click on it, you find in the pre-op you're overstaffed, in post-op you're understaffed," says Mr. Belokrinitsky.
Instead of eliminating or adding staff to match a benchmark, health systems should instead assess and repair their staffing pyramids. This involves measuring the amount of work to be done and determining the right people to execute each of the tasks involved. For some organizations, this might result in the addition of nurses. For others, it might mean bringing on more technicians.
Staffing is complex even without the inherent fluidity of benchmarks that vary from year to year and region to region. Hospital leaders may benefit most from what some might consider an old-school approach. "There is no substitute for standing in the unit with a clipboard and stopwatch, conducting stop-motion studies on how much work needs to be done, who should be doing it, how much can be automated and what processes can be eliminated," says Mr. Belokrinitsky.
Conclusion
Hospital-wide benchmarking comes with a decent amount of caveats and risks. As attractive as a data-driven dashboard might be to overstimulated executives and managers, they should exercise caution and not base staffing and budgetary on benchmarks. ACOs and other performance-based payment models have unleashed even more data, and as a result, organizations are striving to identify the metrics that are meaningful to patient outcomes but also controllable. That balance is important yet difficult to find. Further, to drive meaningful improvement throughout the organization, ACOs and health systems should pair cost and quality data to provide staff with a comprehensive assessment of where the organization is today and where it needs to be next month or next year.