ChatGPT was able to answer 25 questions about heart disease prevention, as well as make recommendations about how to lose weight and lower cholesterol, Politico reported Feb. 16.
The news outlet cited a Feb. 3 study conducted by Cleveland Clinic and Stanford University that analyzed how well ChatGPT could answer healthcare questions regarding heart disease prevention.
The researchers, whose findings were published in JAMA, found that:
- ChatGPT was able to answer 21 of 25 questions correctly.
- ChatGPT responses were considered to be appropriate for a broad range of recommendations, including questions regarding weight loss, how to lower cholesterol and the meaning of certain coronary calcium scores.
- ChatGPT couldn't understand nuanced questions, such as those regarding exercise. It "firmly recommended" cardio and weightlifting, an exercise that could harm some patients, according to the researchers.
- ChatGPT also missed key details on questions about cholesterol levels, and incorrectly suggested that a cardiovascular disease drug isn't commercially available.