SAM-GT: SAM as a General Teacher Enhances Medical Image Segmentation by Distilling Only What Matters

Abstract

The Segment Anything Model (SAM) exhibits strong segmentation and generalization capabilities, attracting significant attention in medical image segmentation. Existing research primarily adapts SAM for medical applications by fine-tuning its decoder or applying parameter-efficient fine-tuning (PEFT) to its encoder, but these paradigms do not take advantage of the structural strengths of specialized medical image segmentation models. To bridge this gap, we propose a novel paradigm that directly distills SAM’s core knowledge into medical segmentation models to further enhance their performance while preserving the strengths of both. However, conventional distillation methods lack knowledge decoupling and selective transfer, limiting their ability to capture SAM’s core knowledge. To address this, we introduce SAM-GT, which distills SAM’s edge knowledge through Edge Knowledge Extraction Distillation (EKED) and conveys its structural priors via Uncertainty-guided Spatial Relation Distillation (USRD). Experiments show that SAM-GT surpasses existing distillation methods, markedly improving state-of-the-art medical segmentation models and achieving performance comparable to medical SAM variants while maintaining a significantly simpler architecture.

Ähnliche Arbeiten