Straggler Remission for Federated Learning via Decentralized Redundant Cayley Tree
Yangyang Tao, Junxiu Zhou · 2020
Federated learning, one of the parallel machine learning models, recently has become a promising direction for large scale mobile edge computing. It leverages the computational power of edge devices (i.e., mobile devices) to train machine learning models with distributed data sets on edge devices without uploading data sets to the cloud. The straightforward benefit is avoiding data security concerns. However, federated learning also encounters challenges, like stragglers (end-devices that respond slowly) among devices due to heterogeneity of devices. Most of the existing body of research tries to address the problem through training objects replication and information coding design such as maximum distance separable (MDS) code. In this paper, we propose a decentralized redundant n-Cayley tree (DRC-tree) for federated learning. Our proposal aims to explore the hierarchical structure of the n-Cayley tree to enhance the redundancy rate in federated learning to mitigate the impact of stragglers. In the DRC- tree structure, the fusion node serves as the root node, while all the worker devices are the intermediate tree nodes and leaves that formulated through a distributed message passing interface. the redundancy of workers is constructed layer by layer with a given redundancy branch degree. The optimality of the proposed architecture is theoretically analyzed and simulated experiments are also conducted through CloudLab with synchronous stochastic gradient descent (SGD) implementation. The experimental results show that the proposed DRC-tree architecture reduces the errors caused by stragglers compared with the state of arts.