One AI Training Fits All? Exploring Behavioral Personas in Rare Cancer Diagnosis— Fumarate Hydratase-Deficient Renal Cell Carcinoma

Abstract

<title>Abstract</title> Fumarate Hydratase-deficient Renal Cell Carcinoma (FHdRCC) is a rare (&lt; 0.4% of RCCs), aggressive subtype with significant morphological overlap, posing diagnostic challenges. While artificial intelligence (AI) shows promise in common cancer diagnostics, its impact on pathologist decision-making in rare diseases—particularly concerning automation bias—remains poorly understood. We developed a deep learning model to classify FHdRCC. We conducted a crossover reader study with 21 pathologists (7 genitourinary (GU) specialists, 7 non-GU specialists, 7 residents) diagnosing 30 challenging cases (15 FHdRCC, 15 non-FHdRCC) with and without AI assistance. We analyzed diagnostic performance and performed an exploratory analysis of human-AI interaction by quantifying AI Acceptance Rate (AAR) and Automation Bias Rate (ABR)—the rates of following AI recommendations when correct or incorrect, respectively—leading to the identification of preliminary behavioral personas. AI assistance significantly improved diagnostic accuracy (60.0% to 73.3%, p = 0.012) and inter-rater reliability (Fleiss' κ from 0.311 to 0.482, p &lt; 0.001). AI-driven gains were negatively correlated with baseline expertise (R=-0.66, p = 0.001), revealing independence from traditional training. Clustering identified two behavioral personas: Receptive (n = 14; high AAR/ABR) employing efficiency-focused strategies, and Resistant (n = 7; low AAR/ABR) using deliberation-focused approaches. While the Receptive group drove accuracy gains (p = 0.041), high automation bias neutralized improvements in overall optimal decision-making (p = 0.259). AI assistance enhances rare cancer diagnostic accuracy, but effectiveness is mediated by behavioral personas rather than traditional expertise. The performance paradox—accuracy gains offset by automation bias—suggests persona-tailored training is essential: Receptive users need critical evaluation skills; Resistant users need trust-building. These exploratory findings require validation in larger studies with sufficient power to characterize automation bias patterns.

Ähnliche Arbeiten