FedGroup-Prune: IoT Device Amicable and Training-Efficient Federated Learning via Combined Group Lasso Sparse Model Pruning
ZiYao Chen, Jialiang Peng, Jiawen Kang, Dusit Tao Niyato · IEEE Internet of Things Journal · 2024
Federated learning (FL) has emerged as a crucial approach in the realm of distributed machine learning, providing a framework for training models on decentralized data while preserving data privacy. This paradigm has established itself as an effective solution for deploying artificial intelligence technology in scenarios associated with the Internet of Things (IoT). Despite its potential, FL faces encounters several challenges, particularly the limited computational and communication capabilities of some local clients, which can hinder further advancement. Such constraints limit the effective implementation and utilization of deep neural networks (DNNs) with numerous parameters on IoT devices. Our study tackles this issue by utilizing Group Lasso for model sparsification and pruning, aimed at lowering the computational and communication demands on IoT devices. Moreover, this article proposes a Group Lasso-enabled FL model pruning strategy specifically tailored for IoT, designed to reduce the size of model parameters, and provides theoretical guarantees of FL convergence. Empirical analysis across multiple models and data sets demonstrates that our method effectively halved the parameters in fully connected layers during federated training. This substantial reduction is achieved with minimal impact on accuracy, thus preserving the integrity of model performance and providing a competitive edge over existing methodologies.