Though a study published Jan. 1 describing the success of Google's DeepMind artificial intelligence at diagnosing breast cancer was largely met with praise and excitement, in a Feb. 28 response, a group of AI researchers criticized the study's lack of transparency.
In the initial study, Google researchers found the DeepMind AI produced fewer false positives and false negatives than an expert panel of six radiologists in making breast cancer diagnoses based on mammograms.
In the response, 19 researchers and the Massive Analysis Quality Control Society board of directors criticized Google's failure to explain in the study how the AI was developed and to release the DeepMind algorithm's code. Doing so, they wrote, undermines the scientific value of the study, since its results can therefore not be reproduced or validated by external groups.
"The failure of McKinney et al. to share key materials and information transforms their work from a scientific publication open to verification into a promotion of a closed technology," the group wrote. "We have high hopes for the utility of AI methods in medicine. Ensuring that these methods meet their potential, however, requires that these studies be reproducible."
The researchers behind the letter represent organizations including Boston-based Brigham and Women's Hospital and Harvard Medical School, Stanford (Calif.) University School of Medicine, Philadelphia-based University of Pennsylvania's Perelman School of Medicine, Baltimore-based Johns Hopkins Bloomberg School of Public Health and more.
Google is planning to submit a response of their own in the journal Nature to address the researchers' concerns, according to Stat.
More articles on AI:
AI provider releases social determinants of health tool
Physician viewpoint: If healthcare AI makes a mistake, who is legally responsible?
AI-powered app provides at-home assessment of coronavirus risk: study