MiniSAM: A Lightweight Segmentation Model with Pruning and Knowledge Distillation
Jiaqi Liu, Yu Wang, Chengji Zhang, Dong Wang, Jianbin Zheng, Jian Hua Zhou, Yang Wang, Xinghao Wang, Yanxia Chai · 2024
In practical applications, the Segment Anything Model (SAM) encounters difficulties in being effectively applied in scenarios with limited computing resources. In view of this, this paper, taking SAM-B as the base model, innovatively proposes MiniSAM, a lightweight variant of SAM. Specifically, firstly, model pruning and knowledge distillation strategies are employed to compress the large-scale image encoder module of SAM, optimizing its consumption of computing resources. Secondly, an object detection module is appended to the original basic structure of SAM, which is capable of automatically generating input box prompts, further enhancing the convenience and practicability of the model in relevant applications. Finally, the mask decoder of MiniSAM is meticulously fine-tuned to better adapt to the overall lightweight model architecture, enabling it to perform relatively well even in the context of limited computing resources. Experimental results demonstrate that in human segmentation tasks, despite having an image encoder with only 4.4% of the parameters of SAM Base (SAM-B) and 0.6% of SAM High-performance (SAM-H), MiniSAM achieves segmentation performance comparable to SAM-B.