Dual Prototype Learning for Few Shot Semantic Segmentation
Wenxuan Li, Shaobo Chen, Chengyi Xiong · IEEE Access · 2024
Few-shot segmentation (FSS) is a challenging task because the same class of targets in the support and query image may have different scale, texture and background information. Prototype learning (PL) is a current mainstream FSS method, which characterizes the interaction between the prototype vector and the query feature. However, the prototype vector commonly based on global average pooling only contains first-order feature information, which is vulnerable to varying appearance of similar target and the diversity of background. Moreover, the auxiliary information of the query image is not fully explored in previous prototype learning methods. In this paper, we propose a dual prototype learning (DPL) based on second-order prototype (SOP) and self-support first-order prototype with constraint mechanism (SSFPC) to improve the FSS performance. The SOP can capture higher-order statistical information, obtaining by averaging the covariance matrix of feature map. The similarity between the first-order support prototype and the first-order self-support query prototype is introduced to boost the adaptability of first-order prototype to query image. The remarkable performance gains on benchmarks (PASCAL-5iand COCO-20i) manifest the effectiveness of our method. Our source code will be available at https://github.com/13ww/DPL.git.