A General Solution for Straggler Effect and Unreliable Communication in Federated Learning
Tianming Zang, Ce Zheng, Shiyao Ma, Chen Sun, Wei Chen · 2023
The straggler effect is the main bottleneck for Federated Learning (FL), where the performance of training is degraded by the slowest member. Another significant problem is unreliable communication, which somehow has been neglected in previous studies. That is, the transmission of local models is not successful every time. In this paper, we find that the problems of straggler effect and unreliable communication are implicitly caused by time divergence of User Equipments (UEs) in each training round. Based on this, we propose our solutions for these two problems and show that our solutions can be merged into a general one: the problem of the straggler effect and unreliable communication can be solved with a simple UE selection method. This method consists of two steps: First, we cluster UEs into several groups based on UEs' physical parameters or performance metrics; Second, in each training round, only UEs from the same group are chosen for FL operation. Full explanations are given why the time divergence is statistically reduced, and therefore it can mitigate the aforementioned two problems. Our solutions are further illustrated with some examples and validated by simulations.