Reinforcement Learning from Human-like Feedback Enhances Semantic Communication with Multimodal LLMs
Xinyu Zhang, Xitao He, Mingkai Chen, Lei Wang · 2024
Semantic communication is an emerging intelligent paradigm that provides some special solutions for the various future applications. With the rise of large language model (LLM), it may enhance well on semantic communication. However, how to balance on the accuracy and the efficiency of compression remains an urgent issue to semantic communication with LLMs. In this paper, we design a multimodal large language models (MLLMs)-enabled semantic communication system specifically designed for image transmission, where the reinforcement learning from human-like feedback (RLHLF) framework is proposed to apply reasoning and self-reflection capabilities of MLLMs to enhance the quality of semantic extraction and image generation. For one thing, we first optimize the image semantic information at the transmitter by RLHLF framework. For another thing, the generated images are optimized at the receiver by RLHLF framework. Simulation results show that our proposed system provides bandwidth savings and enables graceful performance improvement as the channel quality improves, which proves the potential of MLLMs for future semantic communication.