CMS brought about its hospital star rating system as a way to simplify the large quantities of data the agency collects about hospitals and present it in a usable way. It was a simple idea, as the score is based on patient satisfaction and experience determined using HCAHPS scores. But research suggests the ratings only give patients an accurate picture of how hospital stack up against one another when comparing those with similar characteristics.
In a recent article in JAMA, Ashish K. Jha, MD, a Cambridge, Mass.-based Harvard school of public health professor and practicing internist, detailed his efforts, along with colleagues, to determine whether patients who used star ratings to pick hospitals would actually be selecting providers who deliver better care.
"We examined whether, holding all other factors constant, picking a 5-star hospital would lead patients to a hospital with lower mortality hospital than that of a 1-star hospital? It turns out that it does — a lot," Dr. Jha wrote.
And it's a significant effect, he writes. The data bears out that for every 70 patients moved to a five-star hospital from a one-star hospital, a life would be saved. But Dr. Jha's model held certain factors constant, only comparing mortality outcomes among similar hospitals. When he and his colleagues compared star ratings between different kinds of hospitals, such as an urban teaching hospital to a small rural hospital, the usefulness of star ratings dropped off.
"This means that stars aren't a substitute for other information," Dr. Jha wrote. "For example, we know that for certain major conditions, large, teaching hospitals may have better outcomes. Patients shouldn't ignore that. But when choosing among large teaching hospitals in their region, for instance, stars can be helpful."
Dr. Jha concludes that because star ratings rely so heavily on patient-reported information, which is affected by a variety of factors outside of a hospital's control and patient populations vary so greatly, they become a less useful tool to compare, say, a metropolitan teaching hospital to a small rural hospital.
However, Dr. Jha wrote, the current utility of star ratings for patients may improve in the near future, as CMS recently announced plans to introduce a new, more comprehensive star rating system, that will include measures such as mortality, readmissions and timeliness of care, among other factors. While CMS star ratings have been a good step forward in the quest to measure hospital quality, more needs to be done to put the data being shown to consumers in context, Dr. Jha wrote.
"[I]t's worth remembering that when it comes to quality measures, as in so many things in life, more isn't better," Dr. Jha wrote. "Better is better. We need to focus on what we can measure well and, most important, focus on what matters most to patients."
More articles on quality:
CMS delays overall hospital star ratings release: 4 things to know
Seeing stars: CMS quality ratings in skilled nursing facilities
Overall star ratings from CMS still drawing ire, admiration after delay