The study, published March 11 in Communications Medicine, evaluated multiple machine learning models across four datasets for two clinical prediction tasks: in-hospital mortality prediction and five-year prognosis for breast and lung cancer. Researchers found that, for in-hospital mortality prediction, the models failed to recognize 66% of cases involving critically abnormal vital signs and other indicators of severe health decline.
The study also found that machine learning models for five-year survival prediction in breast and lung cancer often fail to respond adequately to worsening health conditions, raising concerns about their reliability in assessing long-term risk.
“Our study found serious deficiencies in the responsiveness of current machine learning models,” Daneng Yao, PhD, said in a news release. “Most of the models we evaluated cannot recognize critical health events and that poses a major problem.”
The findings demonstrate that AI-based models trained solely on patient data are insufficient, researchers said. To improve accuracy and clinical usefulness, models should integrate medical expertise rather than relying solely on statistical patterns.
“A more fundamental design is to incorporate medical knowledge deeply into clinical machine learning models,” Dr. Yao said. “This is highly interdisciplinary work, requiring a large team with both computing and medical expertise.”