The measures frequently used by government agencies and public ranking organizations to rate hospital safety do not accurately summarize the quality of care provided at a facility, according to new research from Baltimore-based Johns Hopkins Armstrong Institute for Patient Safety and Quality.
The authors of the study looked at 19 studies conducted between 1990 and 2015 that focused on the validity of the 21 patient safety indicators and hospital-acquired conditions measured by the Agency for Health Care Research and Quality and CMS. These 21 measures are used by the U.S. News and World Report Best Hospitals, Leapfrog Hospital Safety Score and CMS Star Ratings to evaluate hospitals. To determine the validity of each measure, the researchers compared errors listed on medical records with errors listed in billing data.
Of the 21 measures studied, 16 did not have sufficient data and could not be evaluated for their validity. Examining the five measures with enough information to analyze, only one — the accidental punctures or lacerations obtained during surgery metric — was considered valid by the researchers because the medical record and billing data match more than 80 percent of the time.
The measures studied are based on billing data input from hospital administrators, as opposed to clinical data obtained from patient medical records, which can cause considerable variance in how medical errors are coded from one hospital to another.
"The variation in coding severely limits our ability to count safety events and draw conclusions about the quality of care between hospitals," said Director of the Armstrong Institute Peter Pronovost, MD, PhD. "Patients should have measures that reflect how well we care for patients, not how well we code that care."
Lead author of the study Bradford Winters, MD, PhD, said the measures can potentially misinform patients.
"If the measures don't hold up to the latest science, then we need to re-evaluate whether we should be using them to compare hospitals," concluded Dr. Winters.
More articles on quality measures:
CMS to host webinar on overall hospital star ratings methodology
Number of 5-star hospitals up slightly in new Hospital Compare update
CMS delays overall hospital star ratings release: 4 things to know