Multi-Modal Large Language Model for Medical Auxiliary Diagnosis

Abstract

The application of language models in medicine faces persistent challenges, including outdated knowledge, factual hallucination, and limited integration of multi-modal clinical data. To overcome these issues, this study presents DiagnostiCare, a multi-modal enhanced intelligent diagnostic system designed for medical auxiliary diagnosis. The framework combines a core language understanding model with specialized modules for real-time retrieval (MedRAG), structured knowledge reasoning (MedKG), multi-modal data interpretation (Tools module), and comprehensive information access (Search module). A challenging dataset of complex medical questions was developed to evaluate system performance through expert-based assessments. Experimental results show that DiagnostiCare achieves superior accuracy, treatment relevance, and information reliability compared with general-purpose models. Ablation studies highlight the essential role of each module, particularly in managing multi-modal information. These results demonstrate the potential of a modular, knowledge-driven approach for developing safe and trustworthy AI systems in clinical practice.

Ähnliche Arbeiten