Multimodal Large Language Models in Biomedicine and Healthcare

Abstract

Multimodal Large Language Models (LLMs) are emerging as powerful tools in biomedical research, offering the ability to integrate diverse data types such as medical images, clinical notes, and molecular profiles. Harnessing their full potential requires navigating a complex pipeline encompassing data preparation, model development, and deployment in real-world clinical settings. This review provides a comprehensive analysis of the current state of Multimodal LLMs in biomedicine and healthcare, with a particular focus on those developed for distinct medical applications and domains. We first examine key datasets used to train these models, highlight their strengths and limitations in enabling multimodal medical understanding, and assess their utility in clinical tasks such as diagnostic support, medical report generation, disease prediction and treatment planning. We further discuss critical issues of interpretability, bias and ethics that must be addressed for safe and effective adoption and highlight promising research directions and innovations with the potential to advance biomedical research and contribute to improved patient outcomes. As such, this review equips bioengineers, clinicians and researchers with the foundation needed to harness multimodal LLMs in medicine.

Ähnliche Arbeiten