Waveformer: A Frequency-Aware Transformer-CNN Hybrid for Hyperspectral Image Demosaicing
Junwei Xu, Mengzu Liu, Shilong Zhang, Fangfang Wu, Tao Huang, Weisheng Dong · IEEE Journal of Selected Topics in Signal Processing · 2025
Hyperspectral image (HSI) demosaicing plays a critical role in reconstructing full-resolution HSIs from multispectral filter array (MSFA) acquisitions. Although recent deep-learning approaches based on CNNs and Transformers have achieved significant improvements, a systematic understanding of how these architectures behave across different frequency components remains limited. To address this limitation, we perform a comprehensive frequency-aware analysis with wavelet decomposition, systematically revealing that Transformers excel at modeling low-frequency global structures, whereas CNNs are superior for reconstructing high-frequency details within the unique context of HSI demosaicing. Motivated by this principled insight, we proposeWaveformer, a novel frequency-aware hybrid framework that explicitly decomposes the HSI into low- and high-frequency components using wavelet transform and assigns their reconstruction tasks to optimally suited architectures: Transformers for low-frequency modeling and CNNs for high-frequency enhancement. Furthermore, to counteract ambiguity in traditional attention mechanisms caused by softmax normalization, we introduce aSelf-Calibration Attentionmechanism that adaptively calibrates each token's self-identity across spatial and spectral dimensions, thereby enhancing the model's discriminative power and suppressing interactions from irrelevant tokens. Extensive experiments conducted on multiple benchmark datasets validate the effectiveness of our design. Our method consistently outperforms state-of-the-art baselines, demonstrating strong generalization ability and enhanced reconstruction quality across diverse scenes and spectral configurations.