SRS121 - Analysing the efficacy of large language models in simulating a virtual sarcoma patient

Abstract

Abstract Introduction Large language models (LLMs), a form of artificial intelligence used to process and generate human-like text, have shown promise in simulating virtual patients (VPs), enhancing medical education through realistic dialogue. While their application has expanded across various specialties, their utility in sarcoma care has not been explored. This study aims to evaluate the efficacy of LLMs in creating sarcoma VPs that mirror the lived experiences of real patients. Methods Three LLMs (ChatGPT Plus, Copilot Pro, and Mistral Pro) were prompted to simulate VPs based on three distinct sarcoma cases. Data were drawn from verified patient stories and forums. Each LLM was asked a standard set of questions about patient experience, and their responses were analysed qualitatively. Additionally, validated outcome measures (MSTS and TESS) were used to compare LLM-generated responses to published patient-reported outcomes. Results The LLMs generated emotionally rich and varied VP dialogues. ChatGPT excelled in emotional depth, Copilot in interactivity, and Mistral in clarity and conciseness. All models produced MSTS and TESS scores that closely aligned with published data, indicating good validity. However, TESS scores tended to be slightly lower than real-world values. Discussion LLMs can realistically replicate sarcoma patient experiences, offering a valuable tool for teaching empathy and communication in medical training. Given the rarity and psychological burden of sarcomas, VPs can help bridge educational gaps and prepare trainees for complex patient interactions. This model is scalable for broader educational use and could support the development of LLM-powered training platforms. Further validation in educational settings is recommended.

Ähnliche Arbeiten