Enhancing Large Language Models with Retrieval-Augmented Generation: A Radiology-Specific Approach

Abstract

Retrieval-augmented generation (RAG) is a strategy to improve the performance of large language models (LLMs) by providing an LLM with an updated corpus of knowledge that can be used for answer generation in real time. RAG may improve LLM performance and clinical applicability in radiology by providing citable, up-to-date information without requiring model fine-tuning. In this retrospective study, a radiology-specific RAG system was developed using a vector database of 3689 <i>RadioGraphics</i> articles published from January 1999 to December 2023. Performance of five LLMs with (RAG-Systems) and without RAG on a 192-question radiology examination was compared. RAG significantly improved examination scores for GPT-4 (OpenAI; 81.2% vs 75.5%, <i>P</i> = .04) and Command R+ (Cohere; 70.3% vs 62.0%, <i>P</i> = .02), but not for Claude Opus (Anthropic), Mixtral (Mistral AI), or Gemini 1.5 Pro (Google DeepMind). RAG-Systems performed significantly better than pure LLMs on a 24-question subset directly sourced from <i>RadioGraphics</i> (85% vs 76%, <i>P</i> = .03). RAG-Systems retrieved 21 of 24 (87.5%, <i>P</i> < .001) relevant <i>RadioGraphics</i> references cited in the examination's answer explanations and successfully cited them in 18 of 21 (85.7%, <i>P</i> < .001) outputs. The results suggest that RAG is a promising approach to enhance LLM capabilities for radiology knowledge tasks, providing transparent, domain-specific information retrieval. <b>Keywords:</b> Computer Applications-General (Informatics), Technology Assessment <i>Supplemental material is available for this article.</i> © RSNA, 2025 See also commentary by Mansuri and Gichoya in this issue.

Ähnliche Arbeiten