Physician vs AI chatbot responses to patient questions

Editor's Note

This study, led by researchers at the University of California San Diego, La Jolla, finds that an artificial intelligence (AI) chatbot assistant can provide responses to patient questions that are of comparable quality and empathy to those of physicians.

A total of 195 randomly drawn patients’ medical questions from a social media forum were used to compare physician and chatbot responses to questions.

Among the findings:

• Chatbot responses were preferred to physician responses in 78.6% of 585 evaluations.
• Mean physician responses were significantly shorter than chatbot responses (52 words vs 211 words).
• Chatbot responses were rated to be of significantly higher quality than physician responses, with a 3.6 times higher prevalence of “good” or “very good” quality responses for chatbot.
• Chatbot responses were rated significantly more empathetic than physician responses, with a 9.8 times higher prevalence of “empathetic” or “very empathetic” responses for chatbot.

The researchers conclude that further exploration of this technology is warranted in clinical settings, such as using AI assistants to draft responses to patient questions that physicians can then edit.