On the Convergence Rate of Federated Learning over Unreliable Networks
Chenyuan Feng, Howard Hua Yang, Zihan Chen, Daquan Feng, Zhenzhong Wang, Tony Q. S. Quek · 2021
Considering unreliable wireless networks, the efficiency of training a statistical model via Federated Learning (FL) over wireless networks is studied in this work. The system consists of a server and a swarm of end-user devices that are termed as clients. The server allots the statistical model directly onto the clients for local training, based on the dataset that resides on their devices, periodically collects the trained parameters to produce an improved model, and sends it back to the clients for a new round of local computing. This process repeats for multiple rounds until the global model converges. Due to unreliable wireless channels and limited communication resources, the server can only select a handful of clients for parameter updates at each iteration. Analytical expressions are derived to characterize the FL convergence rate by accounting for the transmission reliability and momentum method. The analyses unveil that in networks with reliable connections, model training can be accelerated by expanding the bandwidth to include more clients in each communication round, but it becomes ineffectual when the connections are unreliable. Moreover, it is also confirmed that adopting momentum in the global aggregation stage of FL improves the convergence rate, and such an approach is more insulated from the effects of communication failures. These theoretical findings are validated by extensive empirical simulations.