Self-Supervised Text-Vision Alignment for Automated Brain MRI Abnormality Detection: A Multicenter Study (ALIGN Study)

Abstract

Purpose To develop a self-supervised text-vision framework to detect abnormalities on brain MRI scans by leveraging free-text neuroradiology reports, eliminating the need for expert-labeled training datasets. Materials and Methods This retrospective and prospective multicenter study included 81 936 brain MRI examinations and corresponding radiology reports for adult patients at two UK National Health Service hospitals from January 2008 to December 2019 for training and internal testing and 1369 prospectively collected examinations between March 2022 and March 2024 from four separate National Health Service hospitals for external testing (ClinicalTrials.gov no. NCT04368481). A neuroradiology language model (NeuroBERT) was trained using self-supervised tasks to generate report embeddings. Convolutional neural networks (one per MRI sequence) were trained to map scans to embeddings by minimizing mean squared error loss. The framework then detected abnormalities in new examinations by scoring scans against query sentences using text-image similarity. Model diagnostic performance was assessed using the area under the receiver operating characteristic curve (AUC). Results The framework achieved an AUC of 0.95 (95% CI: 0.94, 0.97) for normal versus abnormal classification and generalized to external sites with examination-level AUCs of 0.90 (95% CI: 0.86, 0.93) in Bedford, 0.87 (95% CI: 0.83, 0.90) in Nottingham, 0.86 (95% CI: 0.83, 0.90) in Norwich, and 0.85 (95% CI: 0.81, 0.89) in Yeovil. In five zero-shot classification tasks-acute stroke, multiple sclerosis, intracranial hemorrhage, meningioma, and hydrocephalus-the framework achieved a mean AUC of 0.89 (range, 0.77-0.93). For visual-semantic image retrieval, mean precision was 0.84 among the top 15 images across seven pathologies. Conclusion The self-supervised text-vision framework accurately detected brain MRI abnormalities without expert-labeled datasets. Clinical trial registration no. NCT04368481 <b>Keywords:</b> Head and Neck, Unsupervised Learning, Convolutional Neural Network (CNN), Neuroradiology © The Author(s) 2025. Published by the Radiological Society of North America under a CC BY 4.0 license. <i>Supplemental material is available for this article.</i> See also commentary by Ghodasara in this issue.

Ähnliche Arbeiten