Application of Vision Transformers to Multi-task Learning in the LISA 2025 MRI Challenge

Abstract

Transformer-based architectures are increasingly used in medical image analysis to support diverse tasks under a unified framework. Our solution for the LISA 2025 challenge addresses both image quality classification (Task I) and semantic segmentation (Task II). For Task I, we introduced a slice-wise strategy based on a Vision Transformer (ViT) pre-trained on ImageNet. The model processes 3D MRI volumes decomposed into 2D slices, each carrying the original volume’s quality label. Predictions are combined by selecting the maximum value across slices for each quality category. The ViT encoder remained frozen throughout training, with updates limited to the classification layer. In Task II, a UNETR architecture was applied, incorporating encoder weights pre-trained on SAM-Med 3D. Training involved two stages: initial optimization of the decoder with a fixed encoder, followed by full model fine-tuning using Low-Rank Adaptation (LoRA). In the testing stage, Our approach achieved a weighted F1 score of 0.781 for quality assessment, and average Dice scores of 0.58 and 0.81 for hippocampal and basal ganglia segmentation, respectively. These outcomes highlight the flexibility and effectiveness of transformer-based models in multi-task medical image analysis. Our code for Task 1 has been made openly available at https://github.com/RimeT/lisa2025_task1_teamCGP .

Ähnliche Arbeiten