Self-Supervised Video Object Segmentation by Motion-Aware Mask Propagation
Bo Miao, Mohammed Bennamoun, Yongsheng Gao, Ajmal Saeed Mian · 2022 IEEE International Conference on Multimedia and Expo (ICME) · 2022
We propose a self-supervised spatio-temporal matching method, coined Motion-Aware Mask Propagation (MAMP), for video object segmentation. MAMP leverages the frame reconstruction task for training without the need for annotations. During inference, MAMP builds a dynamic memory bank and propagates masks according to our proposed motion-aware spatio-temporal matching module, which is able to handle fast motion and long-term matching scenarios. Evaluation on DAVIS-2017 and YouTube-VOS datasets show that MAMP achieves state-of-the-art performance with stronger generalization ability compared to existing self-supervised methods, i.e., 4.2% higher mean$\mathcal{J}$&$\mathcal{F}$on DAVIS-2017 and 4.85% higher mean$\mathcal{J}$&$\mathcal{F}$on the unseen categories of YouTube-VOS than the nearest competitor. Moreover, MAMP performs at par with many supervised video object segmentation methods. Our code is available at: https://github.com/bo-miao/MAMP.