Federated-Learning-Enabled Cross-Modal Semantic Communication for 6G
Ruochen Huang, Chen Qiu, Mingkai Chen, Changwei Zhang, Hongbo Zhu · IEEE Internet of Things Journal · 2025
In view of super-large scale access and dynamic connectivity requirements in 6G, the number of users and data is increasing exponentially, which makes it difficult to achieve sustainable development of communications. Meanwhile, with the development of cross-modal processing, semantic information is highly considered accurate and intelligent, which is expected to bring new ideas for 6G. Therefore, in this study, we propose a novel framework of cross-modal semantic communications to improve the efficiency of the multimodal processing, especially the tactile modal. Meanwhile, we have redesigned the significant aspects of artificial intelligence (AI), such as encoding, transmission, and processing. As federated learning (FL) inherently supports multiple privacy-preserving and security measures, we also introduce FL to assist AI training in cross-modal semantic communications, including the expansion of multimodal data, cross-modal semantic extraction, comprehensive decision-making, and privacy protection. First, in encoding aspects, we propose a hybrid coding for haptic signal coding by deep learning (DL) according to the semantic association between material identification and tactile code optimization. Second, in transmission aspects, we propose a modal-aware resource allocation for the fairness optimization between transmission requirements and network resources with deep reinforcement learning (DRL). Third, in signal processing, low-rank signal reconstruction and immersive quality of experience (QoE) evaluation by DL and machine learning (ML) are provided to improve and quantify users’ experience. In addition, the computational experiments of those key technologies are shown individually and the specification of their probability is discussed separately. Finally, future research topics related to the above issues are suggested.