Mamba-SAM: An Adaption Framework for Accurate Medical Image Segmentation
Yifeng Wu, Xiaodong Zhang, Haoran Zhang, Yang Shan Sun, Lin Li, Fengjun Zhu, Dezhi Cao, Jinping Xu · 2024
The Segment Anything Model (SAM) shows strong performance in natural images but struggles with medical images due to a significant semantic gap and characteristics like heterogeneity, low contrast, and individual variation. To tackle these challenges, the Mamba-SAM adaption framework is introduced, utilizing pre-trained SAM as a foundation. It integrates two novel modules: the Low-Rank Adaption module, which fine-tunes the image encoder with fewer trainable parameters to bridge the semantic gap, and the Encoding Mamba Adapter module, designed to learn global task-related features at various levels. These features are fused in the mask decoder for effective medical segmentation. Extensive experiments on two public lesion segmentation datasets demonstrate that Mamba-SAM outperforms existing segmentation methods both quantitatively and qualitatively.