Hierarchical Signal Calibration and Refinement for Multimodal Sentiment Analysis
Baojian Ren, Tao Cao, Zhengyang Zhang, Shuchen Bai, Na Liu · IEEE Signal Processing Letters · 2025
To address the issues of noise amplification and feature incompatibility arising from modal heterogeneity in multimodal sentiment analysis, this paper proposes a hierarchical optimization framework. In the first stage, we introduce the Semantic-Guided Calibration Network (SGC-Net), which, through a Dynamic Balancing Regulator (DBR), leverages textual semantics to intelligently weight and calibrate the cross-modal interactions of audio and video, thereby suppressing noise while preserving key dynamics. In the second stage, the Synergistic Refinement Fusion Module (SRF-Module) performs a deep refinement of the fused multi-source features. This module employs a Saliency-Gated Complementor (SGC) to rigorously filter and exchange effective information across streams, ultimately achieving feature de-redundancy and strong complementarity. Extensive experiments on the CMU-MOSI and CMU- MOSEI datasets validate the effectiveness of our method, with the model achieving state-of-the-art performance on key metrics such as binary accuracy (Acc-2: 86.73% on MOSI, 86.52% on MOSEI) and seven-class accuracy (Acc- 7: 48.35% on MOSI, 53.81% on MOSEI).