Artificial intelligence is on par with physicians for virtual primary care diagnoses, according to a new study in Mayo Clinic Proceedings: Digital Health.
The researchers analyzed 102,059 virtual primary care visits between October 2022 and January 2023 in which patients started with an AI interview before being diagnosed by a physician or advanced practice provider. Providers agreed with an AI diagnosis in 84.2 percent of cases and the top-ranked AI diagnosis in 60.9 percent of them, according to the Sept. 20 study. Independent clinician adjudicators' diagnoses lined up with the AI as well.
"This pioneering study marks the first comprehensive examination of AI's performance versus physicians in real-life cases as opposed to textbook vignettes," said lead author, Dan Zeltzer, PhD, a researcher at Israel's Tel Aviv University, in a Sept. 20 news release. "Beyond its high accuracy, this AI stands out for its absence of hallucinations, errors and impartiality across demographics."
The AI and clinicians were also in more than 90 percent agreement on their differential diagnoses for 57 of the most common primary care conditions.
The visits used the 24/7 virtual primary care platform from startup K Health, in which AI interviews patients for an average of five minutes and 25 questions, leading to an average of six positive symptoms and 2.5 potential diagnoses. Providers then review the interview summary, communicate with patients virtually and finalize a diagnosis and treatment plan.
K Health has trained its AI on deidentified patient data from Mayo Clinic Platform, the data and digital health arm of the Rochester, Minn.-based health system. The AI learns from the data and clinical supervision, improving its predictive algorithms.