Multiscale Superpixel Affinity for Improved Weakly Supervised Semantic Segmentation
Yun Wei Fu, Wenwu Wang, Lei Zhu · 2024
Weakly supervised semantic segmentation based on image-level labels typically employs strategies of modifying and expanding class activation map(CAM) seeds to achieve semantic segmentation. However, network downsampling can reduce the edge perception of the CAM, leading to problems of under-activation and over-activation, which consequently affect the segmentation quality. To address these issues, this paper proposes a weakly supervised semantic segmentation method based on multi-scale superpixel affinity(MSPA). In this method, a classification network is first used to generate CAMs, which are then refined using a pixel adaptive module to generate pseudo-masks and provide semantic labels for superpixels. Simultaneously, a fully convolutional network is utilized to extract full-size image features, which are mapped onto the superpixel segmentation map to obtain the feature representation of each superpixel. Subsequently, based on the feature representation of superpixels, the affinity between superpixels is calculated. The superpixels associated with foreground and background categories are then clustered separately to generate foreground probability maps and background probability maps to supervise the localisation of the CAMs. Experimental results demonstrate that our method achieves segmentation accuracies of 67.02% and 67.74% on the validation set and test set, respectively, on the Pascal VOC 2012 dataset. This leads to more accurate activation of the CAMs and clearer categories, which significantly outperforms the compared methods in this paper.