Adaptive Clustering-Enabled Large-Scale Decentralized Federated Learning
Xuang Liang, Jianhua Tang, Marie Siew, Tony Q. S. Quek · 2024
Since there exists a single point of server failure in conventional centralized federated learning, the decentralized federated learning (DFL) framework has become increasingly popular in recent years. However, when a large number of edge devices participate in DFL, it requires frequent model interactions between edge devices and long convergence time. In this work, we combat the impact of device heterogeneity in the large-scale DFL framework. To improve communication efficiency between edge devices, we propose a decentralized edge devices clustering (DEDC) approach to adaptively group edge devices with dense connectivity and similar data distributions into one cluster and form a novel multi-cluster decentralized federated edge learning (MD-FEEL) framework. We propose an asynchronous algorithm in the formed MD-FEEL framework, which consists four steps, i.e., local stochastic gradient descent (SGD) update, gradient consensus, intra-cluster model aggregation and inter-cluster model aggregation. We prove the convergence of our proposed asynchronous MD-FEEL algorithm on a non-convex setting and elaborate on the effect of some hyperparameters. Empirically, we evaluate our proposed asynchronous MD-FEEL on the MNIST and CIFAR-10 datasets. The simulations show that our proposed asynchronous MD-FEEL can perform better in terms of convergence speed and generalization performance than some benchmark algorithms.