Clinical usability of generative artificial intelligence for MR safety advice

Abstract

PurposeThis exploratory, proof-of-concept study comparing the clinical usability of answers to MR Safety queries from two generative AI models and a human MR Safety Expert, assessed by expert review panel. MethodsThis study is a mixed retrospective-prospective, proof-of-concept study.A clinical MR safety advice archive (January 2024 to April 2025) was used to curate 30 generic MR safety support requests with associated MRSE responses.ChatGPT-4o (ChatGPT) and Google AI Overview (GAIO) were prompted with these generic requests to generate AI safety advice.An expert panel assessed all answers for clinical usability.Unusable responses were assigned as; "Unsafe Advice", "Safe but Incorrect", "Incomplete Advice/ Key Details Missing", "Contradictory Statements"," Out of Date".Requests were subcategorised into "specific" and "generic" requests, as well as "passive" and "active" implants, and "other" requests for post review analysis. ResultsPercentages of usable answers and reasons for non-usable responses were compared.Overall, 93% (28/30) of the human responses, 50% (15/30) of the GAIO responses and 43% (13/30) of the ChatGPT responses were deemed acceptable for clinical use.Subcategorization usability was: "generic"; Human 94% (16/17), GAIO and ChatGPT 59% (10/17), "specific": Human 92% (12/13), GAIO 38% (5/13), ChatGPT 23% (3/13), Active: Human 100% (9/9), GAIO 33% (3/9), ChatGPT 22% (2/9), "passive": Human 88% (14/16), GAIO 56% (9/16), ChatGPT 50% (8/16) and "other"; Human 100% (5/5), GAIO and ChatGPT 60% (3/5).While both AIs were able provide clinically acceptable answers for some requests, they did so at a markedly lower success rate than a human MRSE.

Ähnliche Arbeiten