FedCPSO: Federated Learning with Combined Particle Swarm Optimization
Hongjian Shi, Ruhui Ma, Haibing Guan, Weishan Zhang · 2023
The development of the Internet of Things (IoT) has allowed devices to collect massive amounts of data, and Artificial Intelligence (AI) provides the ability to analyze those data. Moreover, researchers adopt Distributed Machine Learning (DML) methods to train neural networks collaboratively using different users' data. However, DML suffers from privacy issues, and Federated Learning (FL) has been an effective solution. FL transfers the model instead of the data to protect privacy, but the trained models have low accuracies over local datasets due to statistical heterogeneity. Thus, personalized FL (pFL) algorithms have been proposed to handle such heterogeneous data distribution. However, the communication overhead in the pFL algorithms is significant as it requires transmitting additional information. Thus, we propose Federated Learning with Com-bined Particle Swarm Optimization (FedCPSO) in this paper. FedCPSO replaces the aggregation process of FL algorithms with PSO, and we design a velocity in PSO specifically for FL algorithms, using the best global model, the best client models, and the best neighbor models. In addition, we also implement magnitude pruning to reduce the communication volume. The experimental results illustrate that FedCPSO can reduce up to 50% communication volume while having less than a 2% accuracy drop compared with the State-of-the-art (SOTA) pFL algorithm.