A new study suggests large language models have the potential to make hospital quality reporting simpler and more efficient, significantly reducing the time spent on the task.
A pilot study led by researchers at UC San Diego Health found an advanced AI system using LLMs achieved 90% agreement with manual quality reporting measures. The team tested the system on a sample of 100 abstractions for the CMS SEP-1 measure for severe sepsis and shock, finding the model achieved agreement with manual abstractions in 90 of the 100 abstractions included in the study.
The findings suggest that LLMs can perform accurate abstractions for complex quality measures that typically require meticulous analysis of patient charts. The abstraction process for SEP-1, for example, typically involves a 63-step evaluation that requires weeks of effort from reviewers.
"We remain diligent on our path to leverage technologies to help reduce the administrative burden of healthcare and, in turn, enable our quality improvement specialists to spend more time supporting the exceptional care our medical teams provide," Chad VanDenBerg, study co-author and chief quality and patient safety officer at the health system, said in a news release.