Adaptive Lightweight Semantic Communications Empowered Model Training Framework for IoT Multimodal Inferencing Businesses
Yonghao Qi, Siya Xu, Qi Feng, Peng Yu, Weiwei Shi · 2025
With the rapid development of the Internet of Things (IoT), multimodal data is increasingly crucial in various tasks. Semantic communication and Federated Learning (FL) utilize this data for model training, addressing communication overhead and privacy protection. However, IoT devices’ resource constraints necessitate optimizing model structures by reducing parameter counts. This paper introduces a two-stage distributed training framework for multimodal semantic models (MSM-TSDTF), combining FL training with model lightweighting. Initially, high-capacity devices perform pre-lightweighting of semantic models. Subsequently, a model adaptive lightweighting algorithm optimizes the model structure during the distributed pruning stage, enhancing the FL process. Furthermore, this paper proposes a network pruning and quantization-based model adaptive lightweighting algorithm (NPQ-MALA), which considers weight contributions and sparsity ratio impacts on model accuracy. This algorithm restores mis-pruned weights through pruning-quantization optimization, achieving model adaptive lightweighting. Simulation results show that MSM-TSDTF significantly reduces training latency and energy consumption compared to traditional FL-based pruning methods, while NPQ-MALA maintains accuracy comparable to original models.