ChatGPT fails American College of Gastroenterology self-assessment tests

Editor's Note

In this study led by researchers at Northwell Health, New Hyde Park, New York, ChatGPT-3 and ChatGPT-4 did not pass the 2022 and 2021 American College of Gastroenterology self-assessment tests.

The tests help students gauge how they would perform on the actual American Board of Internal Medicine Gastroenterology examination.

ChatGPT-3 scored a 65.1% on 455 questions, and GPT-4 scored 62.4%. A score of 70% or higher was required to pass the test.

The researchers concluded that ChatGPT should not be used for medical education in gastroenterology in its current form.