ChatGPT as an Assessment Tool for MBBS Physiology Exams: A Comparative Study on MCQ and SAQ Answering Efficiency

Abstract

INTRODUCTION: ChatGPT, a language model, is well-known for its capacity to generate human-like responses, but its use in medical education, particularly in assessment contexts, is underexplored. The aim of this study was to evaluate the efficiency of ChatGPT as an assessment tool in medical physiology examinations by comparing its performance in answering MCQs and SAQs. The findings of this study may impact the use of AI in medical education in a constantly digitised academic environment. MATERIALS AND METHODS: The study evaluated the performance of ChatGPT in answering 30 multiple-choice questions (MCQs) and 12 short-answer questions (SAQs) from each of the four physiology blocks. The questions were chosen from previous block exams to ascertain consistency. Two independent evaluators assessed the correctness and relevance of responses from ChatGPT using the answer key. The mean marks obtained by first-year medical students for 120 MCQs and 48 SAQs were compared with those of ChatGPT. RESULTS: ChatGPT performed better than first-year medical students in MCQs in all block exams and the difference in marks was statistically significant in blocks 1, 2, and 3. In SAQs, ChatGPT also performed better than the students in most questions. Students scored better in SAQ 11 in block 2, SAQ 12 in block 3 and SAQ 1, 2, 5 in block 3. CONCLUSION: ChatGPT is an effective AI tool for answering medical physiology questions. However, its performance varies across some MCQs and SAQs, indicating potential limitations in reasoning, contextual interpretation, and application-based problem-solving.

Ähnliche Arbeiten