Large Language Model and Medical Education: Evaluation of Human and Artificial Intelligence Responses to Thoracic Surgery Questions

Abstract

Objective: This study aimed to evaluate the performance of ChatGPT-4, a large language model, in answering thoracic surgery questions compared to 5th-year medical students.The goal was to assess the potential of ChatGPT-4 as an educational tool in medical training.Methods: A retrospective comparative analysis was conducted involving 10 fifth-year medical students and ChatGPT-4.Each participant answered 40 multiple-choice questions related to thoracic surgery.The students' scores were compared to the scores generated by ChatGPT-4.Statistical analysis was performed using an independent sample t-test to determine the significance of the differences in performance. Results:The students' scores ranged from 80% to 97.5%, with an average score of 88.25% (SD=5.63).ChatGPT-4 scored 95% on the same set of questions.The t-test results indicated a statistically significant difference between the students' scores and ChatGPT-4's score (t=-3.98,p=0.00088). Conclusion:The study demonstrated that ChatGPT-4 can provide accurate answers to thoracic surgery questions, surpassing the performance of 5th-year medical students.This indicates the potential of large language models as valuable educational tools in medical training.However, further research is needed to evaluate the model's performance across different medical disciplines and question types.

Ähnliche Arbeiten