FairFusionNet: A Novel Deep Learning Framework for Melanoma Diagnosis using Multimodal Data Fusion

Abstract

Cutaneous melanoma is the most lethal form of skin cancer, necessitating early and accurate diagnosis. While deep learning models applied to dermoscopic images have shown remarkable success, their performance can be limited by intrinsic ambiguities in visual data. Therefore, this research proposes FairFusionNet, a novel deep learning framework designed to enhance diagnostic accuracy and robustness by fusing multimodal dermatological data. FairFusionNet integrates dermoscopic images with patient-level clinical metadata such as age, sex and lesion location and dermatologistdescribed semantic features such as asymmetry and pigment network, within a unified architecture. The core innovation is a hierarchical, cross-modal attention fusion mechanism that dynamically weights the contribution of each modality, giving precedence to the most informative features for a given case. The evaluation of FairFusionNet is performed on the publicly available SIIM-ISIC Melanoma Classification dataset. The results demonstrate that FairFusionNet significantly outperforms models that utilize a single modality, such that the AUC of image-only model is 0.890 and the AUC of multimodal is 0.945. Furthermore, by explicitly incorporating clinically relevant features, the model provides enhanced interpretability, aligning computer-aided diagnosis with established clinical reasoning. The proposed framework represents a significant step towards robust, transparent, and clinically deployable AI tools for melanoma screening.

Ähnliche Arbeiten