Efficient Yet Effective: A Dynamic Self-Distillation Framework for Remote Sensing Image–Text Retrieval
Qimin Cheng, Zilei Zhou, Linfeng Yuan, Yingjie Du · IEEE Geoscience and Remote Sensing Letters · 2025
Remote Sensing Image-Text Retrieval (RSITR) is essential for bridging the gap between heterogeneous data modalities. Recently, Vision-Language Models such as CLIP have gained popularity and become a dominant paradigm in this field. However, the high semantic similarity in remote sensing scenes introduces substantial noise into the binary supervision of contrastive learning, making domain adaptation difficult. Existing methods mitigate this by introducing large-scale additional data, but at the expense of significant training cost. To address this, we propose DSD-RSITR, a Dynamic Self-Distillation Framework that improves learning efficiency and reduces reliance on extensive data. It maintains teacher encoders for both modalities to retain prior knowledge and predict semantic relations before alignment. A dynamic EMA strategy and adaptive distillation loss support rapid teacher refinement in early training stages and stronger supervision as training progresses. Experiments show that DSD-RSITR surpasses RemoteCLIP by 1.55% and 0.29% mR on RSICD and RSITMD, respectively, despite using only 4.7% and 2.1% of the training data. Code is available at: https://github.com/zzl0107/DSD-RSITR.