CineScribe: AI-driven standardization of cardiac motion reports in cine MRI

Abstract

Abstract Background Up to 30% of radiology reports contain ambiguous terms, leading to inconsistent interpretations that impact clinical decision-making. This issue is particularly evident in left ventricular regional wall motion abnormality (RWMA) analysis, where subjective assessment causes variability, even in cine cardiac magnetic resonance (CMR), the gold-standard for RWMA evaluation. Standardizing RWMA reporting improves diagnostic consistency and secondary data usability for research and AI applications. Large Language Models (LLMs) offer a powerful tool for clinical report structuring. Purpose Develop an AI model to extract RWMA diagnoses and generate structured bullseye diagrams. Identify and flag ambiguous reports prone to multiple interpretations. Generate precise, unambiguous reformulations for improved diagnostic clarity. Methods CineScribe, a fine-tuned Llama 3-based LLM, was trained on 346 free-text RWMA reports and their manually annotated bullseye diagrams from short-axis cine CMR studies. The same dataset was benchmarked against Zero-shot GPT-4o for structuration accuracy, measured against expert annotations. Reports were classified as ambiguous if they allowed ≥2 plausible interpretations, as determined by three CMR experts. A CineScribe confidence score, a probabilistic measure of text ambiguity, was introduced and validated for flagging ambiguous reports. CineScribe was tested on 700 additional cine CMR reports, flagging those with confidence scores <0.8 for expert review. The model generated three alternative bullseye diagrams, ranked by confidence, which experts validated alongside cine MRI videos to determine the most accurate diagnosis. Finally, CineCorrect, a secondary model, converted structured diagrams into clear, unambiguous free-text descriptions. Results -CineScribe structured RWMA diagnoses across 16 myocardial segments with five motion abnormality types. -CineScribe outperformed GPT-4o (accuracy: 0.91 vs. 0.71) for RWMA extraction. -The CineScribe confidence score reliably flagged ambiguous reports, achieving >95% agreemen with expert assessment at a 0.80 threshold. -22% of training reports (77/346) were flagged as ambiguous and revised by experts. -CineCorrect successfully generated unambiguous free-text versions, improving clarity. Conclusion CineScribe automates RWMA report structuring, transforming free-text descriptions into standardized bullseye diagrams. It detects and flags ambiguous reports using the CineScribe confidence score and refines flagged cases with CineCorrect, ensuring clarity and consistency. By standardizing reports, detecting ambiguity, and enhancing text precision, CineScribe streamlines workflows, improves diagnostic accuracy, and facilitates AI-driven cardiac motion analysis.CineScribe Report Refinement Pipeline CineScribe vs. GPT-4o performance

Ähnliche Arbeiten