127P AI-powered assessment of pancreatic cancer resectability: A comparative study of chatbot models versus NCCN criteria and radiology reports

Abstract

Background: Resectability assessment in pancreatic cancer is pivotal for treatment planning.While NCCN criteria provide standardization, radiologist interpretations vary.This study evaluated large language models (LLMs), including ChatGPT, for classifying resectability from radiology reports versus NCCN 2025 criteria and expert review. Methods:In this retrospective study, we analyzed 75 patients with histologically confirmed pancreatic cancer and complete baseline imaging (CT/MRI).Resectability was classified into three categories (resectable, borderline resectable, unresectable) using: 1. Manual classification via NCCN 2025 criteria 2. Consensus among gastrointestinal radiologists, 3. Outputs from AI chatbots (ChatGPT-4.5, GPT-4o [o3], o4-mini, and Gemini).Results: Among 75 patients with confirmed pancreatic cancer, resectability was assessed via NCCN 2025 criteria, radiologist consensus, and four AI chatbot models.ChatGPT-4.5 demonstrated the highest concordance with NCCN classification (Cohen's = 0.82) and radiologist consensus ( = 0.84).GPT-4o showed substantial agreement ( = 0.76 and 0.78, respectively), while Gemini and o4-mini had slightly lower concordance.Table: 127P Concordance of chatbots with NCCN and radiologist consensus Model with NCCN with radiologists Agreement ChatGPT-4.5 0.82 0.84 Almost perfect GPT-4o (o3) 0.76 0.78 Substantial Gemini 0.71 0.73 Substantial o4-mini 0.68 0.69 Moderate Discordant cases (n=12, 16%) primarily involved borderline resectable tumors with variable descriptions of vascular involvement (especially SMA, PV, and SMV abutment).In subgroup analysis, interobserver variability was highest in the interpretation of borderline tumors ( range: 0.52-0.65)and lowest in clearly unresectable cases ( >0.85 across all models).Conclusions: ChatGPT-4.5 and comparable AI models demonstrate near-expert performance in pancreatic cancer resectability classification based on radiology reports.These models have potential to support multidisciplinary oncology workflows and reduce variability in clinical decision-making.

Ähnliche Arbeiten