CFedPer: Clustered Federated Learning with Two-Stages Optimization for Personalization
Zhipeng Gao, Yan T. Yang, Chen Zhao, Zijia Mo · 2022
Federated learning(FL) is a privacy-preserving dis-tributed learning paradigm in which clients cooperate with each other to train a global model. It is becoming progressively prevalent with the rapid development of edge devices. A critical challenge in federated learning is the data heterogeneity among clients, resulting in the global model generated by standard federated learning being unable to be adapted to all clients. To tackle this problem, we propose the CFedPer for personalized FL, which generates a personalized model for each cluster after clustering to address the deficiency of standard federated learning. Our algorithm is organized into two optimization phases. The pre-start phase clusters clients by our proposed similarity-based clustering model using distribution vector and similarity matrix. In the in-training phase, we represent the neural network as the base layer and personalization layer and propose a novel optimization objective with a regularization term for the personalization layer to achieve a balance between per-sonalization and generalization, preventing over-personalization. Extensive experiments on various datasets and data distributions indicate that the performance of our algorithm is superior to the existing algorithms in terms of average local accuracy and variance among clients.