Stochastic Gradients Averaging for Federated Learning Algorithm with Compressed Quantization
Jia Wang, Jueyou Li, Yuqi Hao · 2023
Federated learning presents a promising avenue for training machine learning models across distributed edge networks. Given the inherent data heterogeneity and limited communication bandwidth stemming from the multitude of devices involved, the imperative of curtailing communication costs becomes apparent. While the compressed transmission model stands as a prevalent technique, it solely accommodates uniform data distributions, neglecting the influence of heterogeneous data. This study addresses the challenge by compressing the communication model between devices and the server, grounded in the implementation of stochastic gradient averaging to mitigate errors arising from data heterogeneity. Moreover, the algorithm’s convergence analysis is then provided for non-convex loss functions. The results show that our algorithm has the same convergence order as the stochastic controlled averaging federated learning algorithm. We validate our proposed algorithm using the MNIST and the CIFAR-100 datasets to determine its effectiveness.