Optical Flow Guided Pyramid Network for Video Salient Object Detection
Tinglong Tang, Sheng Hua, Shuifa Sun, Yirong Wu, Yuqi Zhu, Chonghao Yue · 2024
Video Salient Object Detection (VSOD) is a significant pre-work for many vision applications. Different for Salient Object Detection (SOD), an effective VSOD model requires not only the spatial domain of origin image but also temporal domain. In this paper, we proposed an optical flow guided pyramid network (OFPN) for VSOD, which exploit the temporal optical flow (OF) to assist VSOD. Due to the fact that optical flow maps have slightly lower quality compared to depth maps, we designed two modules for seeking better improvement. To this end, we render optical flow maps from RGB images firstly. Then, an adaptive cross-modal attention module (ACA) is designed for multi-modal fusion. The high-level encoded features are aggregated into a shared decoder for primary prediction. Besides, the low-level features are separately sent into multi-scale context attention module (MCA) for multi-scale context fusion with the assist of the primary prediction level by level. Further, we exploit a multi-scale loss to take full advantage of the hierarchical details through image pyramid structure. Extensive experiments on five benchmark datasets demonstrate the superiority of our method against 12 state-of-the-art methods.