Researchers from Stanford University (Calif.) found that clinical notes written by ChatGPT were largely indistinguishable from those written by senior medical residents, MedPage Today reported July 17.
The study, published July 17 in JAMA Internal Medicine, asked 30 internal medicine physicians to grade clinical notes about the history of present illness — four written by residents, one by GPT-3.5 — and the grades differed by less than one point on a 15-point scale, according to the story. The notes written by humans were found to be more detailed, however. The physicians were able to pick which one was written by the artificial intelligence chatbot only 61 percent of the time.
"Large language models like ChatGPT seem to be advanced enough to draft clinical notes at a level that we would want as a clinician reviewing the charts and interpreting the clinical situation," senior author Ashwin Nayak, MD, told the news outlet. "That is pretty exciting because it opens up a whole lot of doors for ways to automate some of the more menial tasks and the documentation tasks that clinicians don't love to do."
The authors called for more research and testing before this type of technology is rolled out in a clinical setting as the study looked at fictional conversations between patients and providers and focused on present illness history, which is only one part of the clinical note, according to the story.