Iteratively refined ChatGPT outperforms clinical mentors in generating high-quality interprofessional education clinical scenarios: a comparative study

Abstract

<title>Abstract</title> <bold>Background</bold> Interprofessional education (IPE) is crucial for fostering teamwork among healthcare professionals. However, its implementation is hampered by the scarcity of multidisciplinary faculty and scheduling conflicts. In response to these issues, existing AI tools such as ChatGPT have struggled to generate high-quality clinical scenarios independently. This study investigates the efficacy of ChatGPT-4o, an advanced version of the artificial intelligence model ChatGPT, enhanced by novel methodological innovations, to overcome these barriers.<bold>Methods</bold> This comparative study assessed clinical scenarios generated by ChatGPT using two different strategies—Standard Prompt and Iterative Refinement—against those crafted by clinical mentors. The Iterative Refinement method, inspired by the clinical scenario generation process itself, involves a cyclic process of evaluation and feedback, closely mimicking clinical case discussions among professionals. Scenarios were evaluated for time efficiency and quality, measured through the Interprofessional Quality Score (IQS). Assessments were blinded and involved multidisciplinary experts and students, with statistical analysis performed using independent samples t-tests and χ² tests.<bold>Results</bold> Scenarios developed using the Iterative Refinement strategy were completed significantly faster than those by clinical mentors and achieved higher or equivalent IQS. Notably, these scenarios matched or surpassed the quality of those created by humans, particularly in areas such as appropriate challenge and student engagement. Conversely, scenarios generated via the Standard Prompt method exhibited lower accuracy and various other deficiencies. Blinded attribution assessments by students further demonstrated that scenarios developed through Iterative Refinement were often indistinguishable from those created by human mentors.<bold>Conclusions</bold> Employing ChatGPT with iterative refinement and role-playing strategies produces clinical scenarios that, in some areas, surpass those developed by clinical mentors. This approach reduces the need for extensive faculty involvement, highlighting AI's potential to closely align with educational standards and significantly improve IPE, especially in resource-limited settings.

Ähnliche Arbeiten