Hybrid RAG-Empowered Multimodal LLM for Secure Data Management in Internet of Medical Things: A Diffusion-Based Contract Approach

Abstract

Secure data management and effective data sharing have become paramount in the rapidly evolving healthcare landscape, especially with the growing demand for the Internet of Medical Things (IoMT) integration. The advent of generative artificial intelligence (GenAI) has further elevated multimodal large language models (MLLMs) as essential tools for managing and optimizing healthcare data in IoMT. MLLMs can handle multimodal inputs and generate different kinds of data by utilizing large-scale training on massive multimodal datasets. Nevertheless, significant challenges remain in developing medical MLLMs, especially security and data freshness concerns, which impact the quality of MLLM outputs. To this end, this article proposes a hybrid Retrieval-Augmented Generation (RAG)-empowered medical MLLM framework for healthcare data management. The proposed framework enables secure data training by utilizing a hierarchical cross-chain design. Furthermore, it improves the output quality of MLLMs by using hybrid RAG that filters different unimodal RAG results using multimodal metrics and integrates these retrieval results as additional inputs for MLLMs. Furthermore, we utilize the age of information (AoI) to indirectly assess the influence of data freshness on MLLMs and apply contract theory to motivate healthcare data stakeholders to disseminate their current data, thereby alleviating information asymmetry in the data-sharing process. Finally, we employ a generative diffusion model-based deep reinforcement learning (DRL) technique to find the optimal contract for efficient data sharing. Numerical results show the effectiveness of the proposed approach in achieving secure and efficient healthcare data management.

Ähnliche Arbeiten