A State-of-the-Art Survey on Local Training Methods in Federated Learning
Michal Staňo, Ladislav Hluchý · 2023
Federated learning is a machine learning paradigm that enables the training of machine learning models on decentralised data while significantly reducing the amount of communication required. This paper comprehensively overviews the five generations of local training methods in federated learning. We discuss the key contributions of each generation and the challenges that remain to be addressed. The first generation of local training methods proposed in the early 2010s was based on heuristic approaches. These methods had no theoretical guarantees, but they were often effective in practice. The second generation of local training methods, which was proposed in the late 2010s, was based on the assumption that the data on the different clients was homogeneous. This assumption allowed for the development of more theoretical guarantees for local training methods. The third generation of local training methods, which was proposed in the early 2020s, relaxed the assumption of homogeneous data and allowed for heterogeneous data. This made local training methods more applicable to real-world scenarios. The fourth generation of local training methods, which was proposed in the mid-2020s, further improved the convergence rate of local training methods. This enabled training machine learning models on federated learning with less communication and less computation. The fifth generation of local training methods, which is still under development, is exploring the use of advanced techniques such as acceleration and quantisation to improve the efficiency of local training methods further. We believe that the five generations of local training methods have made significant progress in the field of federated learning. We believe that the future of federated learning is bright, and we are excited to see what the next generation of local training methods will bring. Research on federated learning with adaptive communication has the potential to improve the efficiency and effectiveness of this machine learning method, which could be a topic for further research.