DEMT: Diffusion-Enhanced Multi-Modal Transformer for Bone Fracture Detection

Abstract

Early and accurate detection of bone fractures is essential for improving clinical outcomes and reducing diagnostic workload in radiology. This study proposes a Diffusion-Enhanced Multi-Modal Transformer (DEMT) designed to jointly analyze X-ray images and patient metadata for robust fracture classification. The model is evaluated using the publicly available Kaggle Human Bone Fractures Image Dataset, which includes multiple anatomical regions. A comprehensive preprocessing pipeline is implemented, incorporating conventional augmentation, diffusion-based augmentation for rare cases, and synthetic metadata generation to enhance data diversity. The dataset is stratified and partitioned into training (85%), validation (10%), and testing (5%) subsets. DEMT integrates a Vision Transformer backbone for image representation with a dedicated metadata encoder, fusing both modalities through a cross-attention mechanism. The framework further incorporates uncertainty quantification via specialized loss functions and produces calibrated confidence intervals to support clinical decision-making. The model is trained in PyTorch using the AdamW optimizer, cosine annealing learning rate scheduling, and early stopping. Experimental results show that DEMT outperforms baseline CNN and ResNet architectures, achieving higher classification accuracy and maintaining low calibration error (ECE ¡ 0.1). DEMT also demonstrates strong performance on rare fracture categories, supported by interpretable attention maps and a clinically oriented decision-support dashboard. These findings highlight the potential of DEMT as a reliable and interpretable tool for real-world radiological assessment.

Ähnliche Arbeiten