EHR vendor Epic's sepsis prediction model was found to be more accurate than other models at higher threshold prediction, but was found to have missed a higher share of true cases and was less timely than other existing sepsis tools, an Aug. 25 study published in JAMA found.
Researchers assessed the validity and timeliness of Epic's sepsis model, the Sepsis Prediction Model, against existing sepsis prediction tools, Systemic Inflammatory Response Syndrome, Sequential Organ Failure Assessment, and Sepsis-Related Organ Failure Assessment.
The study, which used data from 2019 to 2020, found that Epic's model was more accurate at the highest scoring thresholds when it was most confident that a patient had sepsis.
But the model missed a higher share of true cases and was less timely than the Systemic Inflammatory Response Syndrome tool and the Sepsis-Related Organ Failure Assessment tool.
"It appears to predict sepsis long after the clinician has recognized possible sepsis and acted on that suspicion," the researchers wrote.
These findings are similar to that of a June 2021 study published in JAMA Internal Medicine that found that Epic's algorithm missed two-thirds of sepsis cases, rarely found cases medical staff did not notice and frequently issued false alarms.
Epic told Becker's in June 2022 that it had made changes to its sepsis prediction model in a bid to improve its accuracy and make its alerts more meaningful to clinicians.
Epic began the development of its new sepsis predictive model in February 2021 and released it to customers in August. The company told Becker's that this study shows that, of the scoring systems researched, that its sepsis predictive model was the most accurate at predicting sepsis.
"Each organization chooses the threshold score that determines when alerts appear. The JAMA study correctly concludes that using a threshold score several points higher than what most Epic organizations use would result in less timely alerts. By design, a lower threshold would result in more timely alerts," a spokesperson told Becker's.
The company said it is committed to continually improving its models.
"Last fall, we released an updated version of the sepsis predictive model and we are working with our customers to implement it. The live organizations have seen more timely alerts and fewer false positives. The JAMA study does not reflect the performance of our updated model," the spokesperson said.
Editor's note: This article was updated Aug. 31 to add in statements from Epic Systems.