Mistral in Radiology: <scp>AI</scp> ‐Powered Classification of Normal and Abnormal Reports

Abstract

ABSTRACT This study investigates the potential of the Mistral large language model (LLM) to classify radiological reports as normal or abnormal using three techniques: Zero‐Shot Learning (ZSL), Few‐Shot Learning (FSL), and Fine‐Tuning (FT), aiming to optimize radiology workflows and improve clinical decision‐making. The dataset consisted of 124 807 radiology reports from MRI and CT scans conducted between 1 May 2024, and 1 November 2024, at our institution. After applying inclusion and exclusion criteria, 123 296 reports were selected for analysis. The Mistral LLM was tested with ZSL, FSL, and FT techniques. Quantitative metrics, including precision, recall, F1 score, and accuracy, were calculated for each technique. Confusion matrices and qualitative analyses of misclassified cases were also performed. ZSL yielded the lowest performance, with an F1 score of 0.191 for the normal class and an overall accuracy of 0.438, due to a high false‐positive rate. FSL improved accuracy to 0.806 but still showed limitations in classifying normal reports (F1 = 0.404). FT achieved the best results, with F1 scores above 0.98 for both classes and an overall accuracy of 0.998, minimizing false positives and false negatives. Classifying radiological reports as normal or abnormal is crucial for prioritizing urgent cases and optimizing workflows. The Mistral LLM, particularly with Fine‐Tuning, demonstrated strong potential for automating this task, outperforming ZSL and FSL.

Ähnliche Arbeiten