Is DeepSeek Capable of Passing the UK Radiology Fellowship Examinations?

Abstract

Objective: This study evaluates the capability of DeepSeek, a large language model-based Artificial intelligence (AI) system, in passing the UK Fellowship of the Royal College of Radiologists (FRCR) examination by assessing its performance on text-based components. Methods: DeepSeek R1, a publicly available AI chatbot, was tested using standardised prompts on 200 Part 1 physics questions and two sets of 120 single-best-answer questions from Part 2A of the FRCR examination. The AI’s performance was compared against the 2024 FRCR pass marks (57%-75% for Part 1 and 55%-60% for Part 2A). Due to its inability to analyse images, DeepSeek was not assessed on the anatomy or Part 2B components. Results: DeepSeek achieved an accuracy of 82% on the Part 1 physics section and 81.67% and 80% on the two Part 2A papers, surpassing the required pass thresholds for all tested sections. Discussion: These findings demonstrate that DeepSeek possesses substantial knowledge relevant to the FRCR examination and suggest potential applications in radiology education. However, its current inability to process image-based questions limits its applicability in practical radiological assessments. Future advancements integrating image analysis capabilities may enhance its role in radiology training and clinical practice. Conclusion: DeepSeek demonstrates high accuracy in answering text-based FRCR questions, highlighting its potential as an AI-driven educational tool. However, further development is required to enable comprehensive AI integration into radiology training and diagnostic workflows.

Ähnliche Arbeiten