Diagnostic Performance of GPT-4o Compared to Radiology Residents in Emergency Abdominal Tomography Cases

Abstract

<b>Purpose:</b> This study aimed to evaluate the diagnostic performance of GPT-4 Omni (GPT-4o) in emergency abdominal computed tomography (CT) cases compared to radiology residents with varying levels of experience, under conditions that closely mimic real clinical scenarios. <b>Material and Methods:</b> A total of 45 emergency cases were categorized into three levels of difficulty (easy, moderate, and difficult) and evaluated by six radiology residents with varying levels of experience (limited: R1-R2; intermediate: R3-R4; advanced: R5-R6) and GPT-4o. Cases were presented sequentially to both groups with consistent clinical data and CT images. Each case included 4 to 7 CT slice images, resulting in a total of 243 images. The participants were asked to provide the single most likely diagnosis for each case. GPT-4o's CT image interpretation performance without clinical data and hallucination rate were evaluated. <b>Results:</b> Overall diagnostic accuracy rates were 76% for R1-R2, 89% for R3, 82% for R4-R5, 84% for R6, and 82% for GPT-4o. Case difficulty significantly affected the diagnostic accuracy for both the residents and GPT-4o, with accuracy decreasing as case complexity increased (<i>p</i> < 0.001). No statistically significant differences in diagnostic accuracy were found between GPT-4o and the residents, regardless of the experience level or case difficulty (<i>p</i> > 0.05). GPT-4o demonstrated a hallucination rate of 75%. <b>Conclusions:</b> GPT-4o demonstrated a diagnostic accuracy comparable to that of radiology residents in emergency abdominal CT cases. However, its dependence on structured prompts and high hallucination rate indicates the need for further optimization before clinical integration.

Ähnliche Arbeiten