Does concurrent reading with AI lead to more false negative errors for cancers that are not marked by AI?

Abstract

Purpose: To determine if reading digital breast tomosynthesis (DBT) concurrently with an artificial intelligence (AI) system increases the probability of missing a cancer not marked by AI for cancers that the radiologist detected reading without AI. Method We retrospectively analyzed an observer study of radiologists reading with and without an AI system. In that study, there were 260 DBT screening exams (65 containing at least one malignant lesion). Twenty-four radiologists read the cases in two separate sessions (with a 4-week washout period) once without the AI tool and once with AI concurrently (i.e., the AI marks and scores were available immediately upon examining the images). We separated the cases into AI-detected and AI-notDetected and then examined only cases that the radiologist recalled when reading without AI. We determined the fraction of cases from each group that the radiologist recalled when reading with AI; this was done separately for cancer and non-cancer cases. Results: When reading without AI, the readers detected an average of 5.0 of 7 (71%) cancers that were not marked by AI (range 1-7) and 49.8 of 58 (86%) cancers that were marked by AI (range 30-57). When reading with AI concurrently, readers found 3.3 (46%) of the 7 AI-notDetected cancers and agreed with 54.2 (93%) of the 58 AI-detected cancers. Using a two-tailed, paired t-test, this difference (46% vs 93%) was statistically significant (p<<0.00001). Nevertheless, the overall sensitivity increased with concurrent reading compared to reading without AI (77% to 85%). Similarly, for non-cancer cases that were recalled (FP) without AI (47%/26% not-marked/marked by AI), there was a smaller fraction recalled for the not-marked cases (8.1% vs 48%, p<<0.00001). This contributed to an increase in specificity with concurrent reading (63% to 70%). Conclusion: When reading with AI concurrently, radiologists are more likely to miss a cancer when AI fails to mark that cancer. Likewise, radiologists are more likely not to recall a non-cancer case when AI fails to mark a lesion in the case, even though the radiologist recalled the case when reading without AI.

Ähnliche Arbeiten