Some radiologists expressed disappointment with the results of Elon Musk's AI chatbot, Grok, after the billionaire asked X users to submit X-rays, positron emission tomography, MRI and other medical images.
"Try submitting X-ray, PET, MRI or other medical images to Grok for analysis," the tech titan wrote Oct. 29 on X. "This is still early stage, but it is already quite accurate and will become extremely good. Let us know where Grok gets it right or needs work."
Several radiologists answered the call, submitting images and then reviewing Grok's results.
Thibaut Jacques, MD, a French radiologist, submitted an MRI of an adhesive capsulitis of the shoulder and said it was an easy, typical pattern. Grok's analysis was "too generic + no diagnosis," he said.
Laura Heacock, MD, a breast radiologist, deep-learning researcher and associate professor at New York University, submitted a series of breast mammograms, ultrasounds and MRIs through GPT-4 and Grok.
"So how did Grok do on breast radiology? A little better than GPT4v, but not a single diagnosis correct," she wrote in an Oct. 29 X post.
Overall, Grok has "a long way to go," Doug Lake, MD, a radiologist based in Ames, Iowa, said. "These are slam dunk, easy cases and it's fascinating the tech sometimes struggles to recognize the modality and can’' nail down easy diagnoses that a medical student would get."
After departing from OpenAI in 2015, Mr. Musk created his own AI company, xAI, which is behind the Grok program. It is not the only AI that analyzes medical images, but his takes a "crowdsourcing" approach to the development.