4D-aware stereo matching via implicit spectral reconstruction with multi-modal training and RGB-only deployment
Wei Tan, Jialun Li, Jun Hu, Fei Zhang, Xuhui Huang, Ziheng Liu, Julian S. Evans, Hugo Hernandez-Figueroa, Sailing He · Optics Express · 2026
Conventional RGB stereo matching suffers from ambiguities caused by metamerism and weak textures. We demonstrate that implicitly recovered spectral information can enhance depth estimation without requiring spectral sensors at deployment. A reconstruction-to-matching architecture maps RGB inputs to a latent spectral space, injecting material priors into stereo matching. To address data scarcity, we develop a co-aperture system combining a liquid crystal tunable filter with Scheimpflug LiDAR for pixel-aligned multimodal acquisition, and propose a "Measure-and-Complete" strategy using sparse LiDAR to generate dense pseudo-ground truth. Experiments show 4.34% reduction in endpoint error, validating the effectiveness of spectral priors for geometric perception.