P0900 Agreement between six artificial intelligence systems, physicians, and ECCO guidelines in inflammatory bowel disease management

Abstract

Abstract Background Managing Inflammatory Bowel Disease (IBD) requires complex decisions as evidence rapidly evolves. Variation in care may arise where clinicians lack specialist training or guideline familiarity. Artificial intelligence (AI) tools could support standardised, evidence-based decisions. This study compared six AI systems and physician recommendations with ECCO guidelines to assess concordance in IBD management. Methods One hundred and ten IBD cases from two Mexican institutions were analysed using six AI systems (ChatGPT-5o, Gemini, LeChat, Grok, OpenEvidence, and DeepSeek). For each case, therapies proposed by each AI were compared with those prescribed by the gastroenterologist and with ECCO guideline recommendations. Thirteen therapeutic domains were evaluated, including antibiotics, 5-ASA, corticosteroids, biologic and small-molecule therapies, diagnostics, and monitoring. Agreement was measured using Cohen’s κ, Fleiss’ κ, and McNemar’s exact test, stratified by Ulcerative Colitis (UC) and Crohn’s Disease (CD). Results Across all variables, overall concordance with ECCO was high. Perfect agreement (κ = 1.000) occurred for antibiotics, diagnostics, symptom management, surgical consultations, monitoring, and anti-IL-23 therapy. Substantial agreement (κ≈0.6–0.8) was found for 5-ASA and corticosteroids, while anti-TNF and anti-integrin therapies showed moderate to fair alignment (κ≈0.3–0.5). Thiopurines showed slight or no concordance. ChatGPT-5o, Gemini, and OpenEvidence had the highest alignment with ECCO and physician decisions; LeChat, Grok, and DeepSeek showed greater variability, especially in biologic choices. Agreement was higher in UC than CD, particularly for 5-ASA and corticosteroids. Conclusion ChatGPT-5o, Gemini, and OpenEvidence showed the highest alignment with ECCO guidelines and physician decisions, likely reflecting broader access to medical data and stronger integration of evidence-based resources. Consistent agreement across standard management domains such as antibiotics, diagnostics, symptom control, surgical consultations, monitoring, and anti-IL-23 therapies indicates that AI can effectively support guideline adherence and promote consistency in everyday IBD care. However, discrepancies in complex therapeutic areas, particularly biologics and immunomodulators, emphasise the continuing need for clinical expertise, contextual assessment, and shared decision-making. These findings highlight the potential of AI to enhance decision-making, reduce disparities in access to specialised IBD care, and improve treatment quality. Future work should integrate AI with real-time data, enhance transparency, and update models to reflect therapeutic advances, ensuring safe and equitable use in personalised, evidence-based care. References: Raine T, Bonovas S, Doherty G, et al. ECCO Guidelines on Therapeutics in Ulcerative Colitis: Medical Treatment. J Crohns Colitis. 2022;16(1):2–17. Torres J, Bonovas S, Doherty G, et al. ECCO Guidelines on Therapeutics in Crohn’s Disease: Medical Treatment. J Crohns Colitis. 2020;14(1):4–22. Da Rio L, Nardone OM, Danese S, et al. Artificial intelligence and inflammatory bowel disease: where are we going? World J Gastroenterol. 2023;29(3):508–520. Fast D, et al. Autonomous medical evaluation for guideline adherence of large language models. NPJ Digit Med. 2024;7:358. Conflict of interest: Dr. Aillaud, Daniel: No conflict of interest Acosta, Eric: No conflict of interest Cendejas Higuera, Alejandro: No conflict of interest Cantu, Ana Karen: No conflict of interest Cortés, Maria Eugenia: No conflict of interest Hernández, Luis: No conflict of interest Reyes, Monica: No conflict of interest Manzano Cortés, Haire: No conflict of interest

Ähnliche Arbeiten