Artificial intelligence deployed in clinical settings to assist with disease detection could lead to diagnostic errors, as the technologies may tend to look for shortcuts, according to University of Washington researchers.
In a May 31 paper published in Nature Machine Intelligence, researchers at Seattle-based University of Washington examined multiple AI models recently deployed as possible tools for detecting COVID-19 from chest X-rays, according to a news release.
The researchers found that rather than learning medical pathology, the AI models instead rely on shortcut learning to identify associations between medically irrelevant factors and disease status. Shortcut learning is less robust than genuine medical pathology and usually means the model only will work in the hospital in which it was developed.
According to the researchers' analysis, the observed AI models ignored clinically significant indicators and instead used characteristics such as text markers on patient positioning that were specific to each dataset to predict whether someone had COVD-19, according to the news release.
"A physician would generally expect a finding of COVID-19 from an X-ray to be based on specific patterns in the image that reflect disease processes," said co-lead author Alex DeGrave, a doctorate student at UW. "But rather than relying on those patterns, a system using shortcut learning might, for example, judge that someone is elderly and thus infer that they are more likely to have the disease because it is more common in older patients."
While these shortcuts are not "wrong, per se," the association is not expected or transparent, which could lead to misdiagnosis, he concluded.