341P Artificial intelligence-supported early detection of lung cancer from chest x-ray in routine clinical practice: Real-world, multicenter study across Czech hospitals

Abstract

Background:The increasing complexity of clinical guidelines and physicians' time constraints create an urgent need for robust clinical decision support systems.Much hope rests on the use of large language models (LLMs) but their non-deterministic nature poses significant risks.Combining LLMs with deterministic rule-based algorithms represents a promising alternative.This study compares the performance of a specialised urology LLM with a hybrid LLM-rule-based system for prostate cancer risk stratification and treatment selection. Methods:We developed 142 non-metastatic prostate cancer index cases representing all risk groups.Two senior urooncologists independently classified each case according to the EAU 2025 risk groups and identified therapies with "strong" EAU prostate cancer guideline recommendations.These cases were processed through two distinct systems: the "EAU Guidelines Bot", a specialised ChatGPT-based LLM fine-tuned on the EAU 2025 guidelines, and a self-developed hybrid clinical decision support system employing LLM-based (Deepseek-V3) feature extraction from unstructured text followed by deterministic rule-based risk stratification and therapy recommendations.The output of each system was then compared with the expertdefined ground truth.The evaluation occurred in August 2025. Results:The hybrid LLM-rule based system significantly outperformed the specialised LLM across both evaluation metrics (p < 0.05).For risk group classification, the hybrid system achieved 90.8% accuracy with a macro-averaged F1 score of 0.92, exceeding the EAU Bot's performance of 80.3% accuracy and a macro F1 score of 0.81.Similarly, the hybrid approach demonstrated stronger performance in identifying the correct therapies with 90.8% accuracy and a macro F1 score of 0.92, compared to 78.2% accuracy and a macro F1 score of 0.77 for the EAU Bot. Conclusions:Even specialised LLMs lack sufficient accuracy for autonomous clinical deployment in prostate cancer risk stratification and treatment planning.However, when integrated with rule-based engines, LLMs reliably structure medical data, enabling hybrid systems to achieve exceptional accuracy.

Ähnliche Arbeiten