After analyzing The Leapfrog Group's Hospital Safety Grade system, researchers from Ann Arbor-based University of Michigan determined hospitals may receive higher grades if they don't self-report certain scores to the patient safety watchdog group.
The Leapfrog Group assigns patient safety grades to every hospital in the nation. The grades are made up of 30 metrics, including some that hospitals self-report on the voluntary Leapfrog Hospital Survey. If a hospital does not participate in the free, optional survey, its grade is determined solely by other publicly reported outcomes and process data. (Find more information here.)
Part of the Leapfrog Hospital Survey that factors into the Hospital Safety Grade is the hospital's performance on National Quality Forum-endorsed Safe Practices.
"The Safe Practices part of the Hospital Safety Grade is based on whether a hospital self-reports that it has adopted certain protocols, and not whether those protocols are actually followed," Jennifer Meddings, MD, the study's senior author, said in a University of Michigan blog covering the study.
Sign up for our FREE E-Weekly for more coverage like this sent to your inbox
Leah Binder, president and CEO of The Leapfrog Group, says the organization includes Safe Practices in its safety grade as a way to give recognition to hospitals for committing to patient safety. "We want to give hospitals credit when they embark on the path forward on patient safety," she tells Becker's in an interview. "Not every hospital has embarked on that path. Those that do deserve recognition."
Higher grades for lower transparency?
As part of the analysis, Dr. Meddings and the other researchers ran simulations to see how safety grades would change for hospitals that didn't participate in Leapfrog's survey if they instead reported perfect or less-than-perfect performance on the Safe Practices portion. Researchers found hospitals would likely receive lower safety grades if they self-reported low adherence to the Safe Practices as opposed to not reporting anything at all.
"Hospitals that have less-than-perfect performance on Leapfrog's list of protocols get a big hit to their grade, but those with most or all protocols in place are rewarded very little," Shawna Smith, PhD, the study's lead author, explained. "Our simulations show that some hospitals would have had better grades when they didn't report than when they reported imperfect compliance."
Leapfrog's response
Ms. Binder was "flabbergasted" when the study came out and questions why researchers would spend time trying to "strategize how hospitals can 'game' the safety grade," she tells Becker's.
"Lots of hospitals are trying to figure out how they numerically look better," Ms. Binder says. "It just doesn't work. It's too complicated a composite," she says of the grade.
Two aspects of the safety grade make it difficult to manipulate, according to a statement from Leapfrog: it's composed of 30 measures, so there are thousands of potential statistical outcomes that cannot be accurately predicted, and it's graded on a curve.
While the Michigan study, which was published this month in the journal Medical Care, suggests hospitals are disincentivized to be completely transparent with Leapfrog, Ms. Binder says it's just the opposite.
"Hospitals that provide Leapfrog with as much data as possible tend to do better over time," she says. "By giving us as much information [as possible], they minimize the impact of any one particular measure that doesn't look good. … As a general rule, it's a good idea to give us as much information as possible."
As for leaders who may now be questioning if they should complete the Leapfrog survey, Ms. Binder first urges them to "do the right thing."
"Whether you participate in the survey or not, do the right thing," she says. "Focus on your safety, make it a priority, put your patients first. Whether or not you can manipulate this particular grade is irrelevant to your patients. What matters to your patients is that you treat them with respect and put their safety first."