Multi-Modal Knowledge Graph Enhanced StyleGAN-Based Cognitive Semantic Communications for Image Transmissions
Wei Wu, Hangtao Mao, Fuhui Zhou, Tianle Yao, Han Hu, Baoyun Wang, Qihui Wu · 2024
In recent years, image semantic communications (ISC) have gained significant attention as a critical area of semantic communications. However, the majority of existing frameworks for image semantic communications are based on end-to-end deep learning (DL) models, which are often considered black-box processes and have the drawback of lacking interpretability. In this paper, we propose a novel multimodal knowledge graph (MMKG) enhanced StyleGAN-based cognitive semantic communications for image transmissions. Specifically, we utilize a large-scale universal multimodal knowledge graph as semantic knowledge base and design a semantic coding method for images. Additionally, a semantic fusion algorithm is also designed to enable controllable semantic restoration based on the additional semantics provided by the multimodal knowledge graph. Simulation results demonstrate that our proposed method outperforms traditional communication systems in achieving higher semantic reconstruction capability’ especially at low bit rates. The results also highlight the significant contribution of additional semantics provided by the multimodal knowledge graph in facilitating image reconstruction.