Multi-Stage Salient Object Detection in 360° Omnidirectional Image Using Complementary Object-Level Semantic Information
Gang Chen, Feng Shao, Xiongli Chai, Qiuping Jiang, Yo‐Sung Ho · IEEE Transactions on Emerging Topics in Computational Intelligence · 2023
Recently, salient object detection (SOD) for 2D images has been extensively studied. However, due to the complexity of scene and the existence of geometric distortions, research on 360° SOD is still lacking with respect to the wide field-of-view. In this paper, we explore a multi-stage solution for SOD of 360° omnidirectional images, which considers the effects of RGB image and the complementary object-level semantic (OLS) information in locating the objects. Specifically, to effectively concatenate two types of features, we propose a novel Multi-level Feature Fusion and Progressive Aggregation Network (MFFPANet) for accurately detecting the salient objects in 360° omnidirectional images, which is mainly composed of a dynamic complementary feature fusion (DCFF) module and a progressive multi-scale feature aggregation (PMFA) module. First, the OLS and RGB images share the same backbone network for joint learning, and the DCFF module dynamically integrates the hierarchical features from the backbone network. In addition, the PMFA module includes multiple cascaded feature integration modules, which gradually integrate multi-scale features via deep supervision in a progressive manner. Experimental results show that the proposed MFFPANet achieves superior performances on two 360° SOD databases.