Standalone AI Versus AI-Assisted Radiologists in Emergency ICH Detection: A Prospective, Multicenter Diagnostic Accuracy Study

Abstract

<b>Background/Objectives.</b> Intracranial hemorrhages (ICHs) require immediate diagnosis for optimal clinical outcomes. Artificial intelligence (AI) is considered a potential solution for optimizing neuroimaging under conditions of radiologist shortage and increasing workload. This study aimed to directly compare diagnostic effectiveness between standalone AI services and AI-assisted radiologists in detecting ICHs on brain CT. <b>Methods.</b> A prospective, multicenter comparative study was conducted in 67 medical organizations in Moscow over 15+ months (April 2022-December 2024). We analyzed 3409 brain CT studies containing 1101 ICH cases (32.3%). Three commercial AI services with state registration were compared with radiologist conclusions formulated with access to AI results as auxiliary tools. Statistical analysis included McNemar's test for paired data and Cohen's h effect size analysis. <b>Results.</b> Radiologists with AI assistance statistically significantly outperformed AI services across all diagnostic metrics (<i>p</i> < 0.001): sensitivity 98.91% vs. 95.91%, specificity 99.83% vs. 87.35%, and accuracy 99.53% vs. 90.11%. The radiologists' diagnostic odds ratio exceeded that of AI by 323-fold. The critical difference was in false-positive rates: 293 cases for AI vs. 4 for radiologists (73-fold increase). Complete complementarity of ICH misses was observed: all 12 cases undetected by radiologists were identified by AI, while all 45 cases missed by AI were diagnosed by radiologists. Agreement between methods was 89.6% (Cohen's kappa 0.776). <b>Conclusions.</b> Radiologists maintain their role as the gold standard in ICH diagnosis, significantly outperforming AI services. Error complementarity indicates potential for improvement through systematic integration of AI as a "second reader" rather than a primary diagnostic tool. However, the high false-positive rate of standalone AI requires substantial algorithm refinement. The optimal implementation strategy involves using AI as an auxiliary tool within radiologist workflows rather than as an autonomous diagnostic system, with potential for delayed verification protocols to maximize diagnostic sensitivity while managing the false-positive burden.

Ähnliche Arbeiten