Multi-Scale Depth-Aware Unsupervised Domain Adaption in Semantic Segmentation
Congying Xing, Lefei Zhang · 2023
Unsupervised domain adaptation (UDA) for semantic segmentation aims to transfer the domain-invariant knowledge from the labeled source domain to the unlabeled target domain. Leveraging highly relevant tasks as auxiliary tasks has become a common approach to UDA tasks because it contributes to the mutual promotion of the tasks. However, when applying task interactions on a single scale, the model fails to perceive the overall context of the image. To address this issue, we propose a multi-scale depth-aware (Mti-DA) method for domain adaption. In particular, we use the channel attention mechanism to distill the task features and then fuse them with other task features as a complement. The semantic features will better perceive the shape and edges of the objects when they are enhanced by the depth features. We perform task interaction on every scale to deliver the full potential of multi-task learning. Exploiting the depth perception on each scale in the source domain to guide the target domain contributes to enhanced segmentation performance because the complementary relationships of different tasks in the target domain are available. Extensive experiments on two benchmarks (GTA5 to Cityscapes and SYNTHIA to Cityscapes) demonstrate that Mti-DA achieves state-of-the-art performance.