Toward Accurate and Real-Time Binocular Vision: A Lightweight Stereo Matching Network for RGB Stereo Images
Zhong Wu, Hong Zhu, Lili He, Jing Shi · IEEE Sensors Journal · 2023
The existing stereo matching networks with high accuracy generally demand high-computational overhead, making them expensive to deploy and unsuitable for many real-time applications, especially on source-constrained hardware platforms. Some recent works have tried to build lightweight stereo matching networks, but their accuracy and inference speed are far from satisfactory when deployed on source-constrained graphics processing units (GPUs). This article aims to build a lightweight stereo matching network with high accuracy. First, we claim that a reasonable disparity upsampling strategy is crucial for lightweight stereo matching to achieve high accuracy, whose importance is underestimated in the existing works. Accordingly, an efficient and effective disparity upsampling strategy named spatial adaptive disparity shuffle (SADS) is proposed to meet the upsampling requirements of disparities within diverse regions. Second, a novel channel-disparity-mixed attention (CDMA) mechanism is proposed to regularize the high-resolution 4-D compact cost volume. The attention values in CDMA are adaptive across channels, pixels, and disparity candidates, making CDMA suitable for regularizing the compact cost volume. Based on the proposed methods, a lightweight stereo matching network named SADSNet is designed that achieves real-time performance with high accuracy on source-limited hardware platforms (e.g., the NVIDIA Jetson TX2). In addition, SADSNet can be easily scaled up or down to suit different application scenarios according to available computational power. Extensive experimental results on multiple datasets show its clear superiority over state-of-the-art (SOTA) lightweight stereo models and even many large accuracy-oriented models.