ChatGPT’s progress over time: A longitudinal enhancing biostatistical problem-solving in medical education

Abstract

<b>Objective:</b> ChatGPT has been recognised as a potentially transformative tool in higher education by enhancing the teaching and learning process. Cross-sectional evaluations have acknowledged this potential. This study evaluates ChatGPT's performance in solving specific biostatistical problems, focusing on accuracy, stability, and reproducibility, and explores its potential as a reliable educational tool in medical education. <b>Methods:</b> The correlation analysis task from <i>Statistics at Square One</i> by Swinscow and Campbell was chosen for its foundational role in biostatistics. Between October 2023 and March 2024, and July 2024, GPT-3.5 and GPT-4 were tested for accuracy in 12 parameters. <b>Results:</b> A statistically significant change in correct response rates was established in repeated measurements in the period October 2023, March 2024, and July 2024 for GPT-3.5 (Q = 100.99, <i>p</i> < 0.001), GPT-4.0 (Q = 89.55, <i>p</i> < 0.001), respectively. The significant GPT-3.5 improvement was established between March 2024/July 2024 (<i>p</i> = 0.004), and between October 2023 and July 2024 (<i>p</i> = 0.008). The significant GPT-4.0 improvement was established between October 2023 and March 2024 (<i>p</i> = 0.004), and between October 2023 and July 2024 (<i>p</i> = 0.026). <b>Conclusion:</b> Over 9 months, GPT-4 demonstrated rapid and consistent improvements, achieving perfect accuracy by March 2024. Although this study documented ChatGPT's advancement within 9 months, ChatGPT should be positioned as a supplementary tool in higher education classrooms, in the presence of educators, to enhance the learning process.

Ähnliche Arbeiten