Distributed Consensus Problem in Federated Learning Paradigm
Xinping Yan, Yiming Qin, Xiaodong Hu · 2021
Federated learning (FL) framework facilitates more and more applications of deep learning algorithms on the existing network architectures, where the model parameters are aggregated in a centralized manner. However, some of FL participants are often inaccessible, such as in a power shortage or device dormant state. That will force us to explore the probability that the parameter aggregation is operated in an ad hoc manner, which is based on consensus computing. In this paper, we first propose a novel FL paradigm, which supports an ad hoc operation mode for FL participants. Second, a discrete-time dynamic equation and its control law are formulated to satisfy the demands from FL framework, with a quantized caching scheme designed to mask the uncertainties from asynchronous updates and measurement noises. Then, the consensus conditions and the convergence of the consensus protocol are deduced analytically, and a strategy of quantized caching to optimize the convergence speed is provided. Last, the theoretical results are validated by numerical simulations.