From Image Transfer to Object Transfer: Cross-Domain Instance Segmentation Based on Center Point Feature Alignment
Jin Wang, Shunping Ji, Tao Zhang · IEEE Transactions on Geoscience and Remote Sensing · 2023
Remote sensing images can have significant appearance differences due to various factors such as atmospheric conditions, sensor types, seasons, and capture times. Therefore, when applying a pre-trained instance segmentation deep learning model to newly accessed remote sensing images, the model’s performance tends to decrease significantly. Current mainstream image-based or feature-based domain adaptation methods are not designed specifically for the cross-domain instance segmentation problem. These methods attempt to align the whole images, which may not be optimal for instance segmentation tasks. To address this issue, we propose a cross-domain instance segmentation method based on object-level alignment. Instead of aligning the entire images from both datasets, we only align the features of each object instance, particularly the representative center point features. Our approach mainly consists of an improved contour-based instance segmentation model for object-based domain adaptation, an object-pasting enhancement technique based on Fourier domain adaptation (FDA) that effectively reduces the gap between the source and target domains of the object instances, and a self-training strategy that dynamically generates pseudo-labels for iterative model training. Our experiments on cross-domain building instance segmentation demonstrate that the proposed method achieves a 9.5 intersection over union (IoU) improvement over the current best method. Additionally, experiments on a cross-domain close-range dataset involving transfer between simulated and real street images show that our method significantly outperforms the current best method by 6.5 mean average precision (mAP). These results on remote sensing and close-range datasets validate the universality of our approach.