Performance Evaluation of ChatGPT-4o on Korean Physical Therapist Licensing Examination

Abstract

Objective:The objective of this study was to evaluate the performance of ChatGPT-4o on the Korean Physical Therapist Licensing Examination(KPTLE) and to explore its potential as a supplementary tool in physical therapy education and assessment.Design: Quantitative experimental study using publicly available national examination data.Methods: ChatGPT-4o was tested on 960 multiple-choice questions from the 47th to 51st KPTLE, withoutany provision of source or domain-specific information.Correct answer rates and pass/fail status were assessed.Additionally, its performance was compared with that of examinees who passed the same examinations.Statistical analyses included one-sample t-tests, effect size calculationsusing Cohen's d. and percentile rankings.Results: ChatGPT-4o achieved an average correct answer rate of 88.9%, consistently exceeding the passing criteria across all years.Compared to students who passed the same examinations, ChatGPT-4o performed significantly better in overall and subject-specific scores (p 0.001), with large effect sizes (Cohen's d 0.8) and top percentile rankings.Although its performance in the medical law section was relatively poor, the overall results indicated stable and strong performance.Conclusions: ChatGPT-4o demonstrated sufficient domain knowledge to pass the written KPTLE often surpassing human examinee performance in standardized multiple-choice formats.These findings suggest the potential for its use in theoretical education and physical therapy assessments.Additional, further research is required to assess its applicability in testing practical skills and real-world clinical environments, particularly given the limitations inherent in current language models.

Ähnliche Arbeiten