Multi-Modal Semantic Communication System with Multiplicative Combination
Minqi Zhu · 2024
Due to the increasing demand for communication resources in future application scenarios, traditional communication methods may fall short of meeting these requirements. As a result, an increasing number of researchers are focusing on semantic communication. Unlike traditional communication, which focuses solely on data, semantic communication emphasizes the semantics being transmitted, enabling high-quality transmission of compressed data. This article proposes a novel multimodal semantic communication model that employs multiplicative combination for fusing data from different modalities. Multiplicative combination assigns varying weights to different modalities, helping to improve transmission robustness in low SNR environments. We conducted simulation experiments for image and text reconstruction tasks, and the results demonstrate that our model outperforms traditional methods, particularly in low SNR scenarios. This is primarily due to the effective reduction of noise interference through the fusion of different modalities in Diversified Combination.