Evaluating AI performance in answering questions related to thoracic anatomy

Abstract

Introduction: ChatGPT, an AI-enabled conversational language model, holds immense promise for applications across healthcare education, research, and clinical practice. A number of recent reports have highlighted promising outcomes from using ChatGPT for answering multiple-choice questions (MCQs), hinting at a potential reshaping of educational methodologies. Aim: The objective of this particular study was to quantitatively assess the performance of ChatGPT-3.5 and ChatGPT-4 in the context of answering anatomical questions focusing on the Thorax in a Gross Anatomy course for medical students. Methods: The research conducted for this study was focused on a comprehensive examination of ChatGPT-3.5 and ChatGPT-4 capabilities in answering 50 multiple choice questions MCQs designed in USMLE style. These questions were randomly selected from Gross Anatomy course exam database for medical students and reviewed by three independent experts. No questions with images were included in this study. The selected questions had different levels of difficulty. Findings: The analysis revealed that ChatGPT-3.5 exhibited a remarkable ability to answer anatomical MCQs, achieving an average accuracy of 45.6±3.8%, significantly surpassing random guessing at 18.8±4.4%. The subsequent ChatGPT-4 demonstrated a 30% improvement, with an even higher accuracy of 75.6±1.7%. Discussion: ChatGPT demonstrates considerable potential as an engaging educational tool for students immersed in the study of anatomy. Its unique strength lies in its ability to incite student engagement and arouse curiosity through an interactive, conversational mode of answer delivery. Conclusion: It is vital to remember that ChatGPT should not be viewed as a replacement for the pivotal role teachers play in the educational journey. Instead, it should be visualized as an auxiliary instrument poised to enrich the educational experience. Future studies should aim to determine clear, holistic guidelines that illuminate the best ways to leverage and apply ChatGPT within the scope of anatomy instruction.

Ähnliche Arbeiten