FIMD: Fusion-Inspired Modality Distillation for Enhanced MRI Segmentation in Incomplete Multi-Modal Scenarios
Rui Yang, Xiao Wang, Xin Xu · 2024
Magnetic Resonance Imaging (MRI) is an invaluable tool for brain tumor segmentation. However, in clinical practice, certain modalities might be unavailable, leading to potential performance degradation in prediction tasks. According to current implementations, different modalities are treated as independent entities during the feature extraction phase, ignoring their inherent complementary nature. In this paper, inspired by knowledge distillation techniques, we introduce the Fusion-Inspired Modality Distillation (FIMD) framework. FIMD leverages the output from a fused multi-modal model as a teacher to guide the training of individual modalities. Each modality benefits from the collective knowledge of all modalities, enhancing its performance. Our FIMD method, devoid of specific architectural constraints, seamlessly integrates into existing multi-modal brain tumor segmentation frameworks. Notably, extensive experiments on BraTS2020, BraTS2018, and BraTS2015 datasets indicate that FIMD can enhance the performance of current state-of-the-art (SOTA) algorithms on Dice by an average of 0.55-1.60%. Especially in the case of missing modalities, the effectiveness of the FIMD framework in enhancing the robustness and accuracy of brain tumor segmentation is demonstrated.