TFGNet: Frequency-guided saliency detection for complex scenes
Yi Wang, Ruili Wang, Juncheng Liu, Rui Xu, Tianzhu Wang, Feng Hou, Bin Liu, Na Lei · Applied Soft Computing · 2024
Salient object detection (SOD) with accurate boundaries in complex and chaotic natural or social scenes remains a significant challenge. Many edge-aware or/and two-branch models rely on exchanging global and local information between multistage features, which can propagate errors and lead to incorrect predictions. To address this issue, this work explores the fundamental problems in current U-Net architecture-based SOD models from the perspective of image spatial frequency decomposition and synthesis. A concise and efficient Frequency-Guided Network (TFGNet) is proposed that simultaneously learns the boundary details (high-spatial frequency) and inner regions (low-spatial frequency) of salient regions in two separate branches. Each branch utilizes a Multiscale Frequency Feature Enhancement (FFE) module to learn pixel-wise frequency features and a Transformer-based decoder to learn mask-wise frequency features, improving a comprehensive understanding of salient regions. TFGNet eliminates the need to exchange global and local features at intermediate layers of the two branches, thereby reducing interference from erroneous information. A hybrid loss function is also proposed to combine BCE, IoU, and Histogram dissimilarity to ensure pixel accuracy, structural integrity, and frequency distribution consistency between ground truth and predicted saliency maps. Comprehensive evaluations have been conducted on five widely used SOD datasets and one underwater SOD dataset, demonstrating the superior performance of TFGNet compared to state-of-the-art methods. The codes and results are available at https://github.com/yiwangtz/TFGNet . • Saliency detection struggles with boundary identification in complex scenes due to errors in multiscale feature fusion. • TFGNet enhances boundary and inner region detection by learning high and low spatial frequency features separately. • TFGNet employs pixel and mask-level decoders to obtain more comprehensive saliency features. • A histogram dissimilarity loss ensures frequency distribution consistency between ground truth and predicted saliency maps. • TFGNet surpasses leading methods with more accurate and complete boundaries in complex scenes.