A Novel Adaptive Gradient Compression Approach for Communication-Efficient Federated Learning
Wei Yang, Yuan Yang, Xiaobing Dang, Hao Jiang, Yizhe Zhang, Wei Xiang · 2021 China Automation Congress (CAC) · 2021
Federated learning is a recent advance in privacy-preserving deep learning. However, recent surveys show that federated learning still has prohibitive communication costs. Since the communication bandwidth of terminal devices is usually quite limited, improving the communication efficiency is critical to model updating. To solve the above issues, we present a novel adaptive gradient compression algorithm, based on non-Independent, Identically Distributed(non-IID) and unbalanced data distributions in federated learning to provide a unique compression rate for each client. Through comparative experiments, we prove that the performance of the adaptive gradient compression algorithm in federated learning is superior to the fixed compression rate method, which significantly improves the communication efficiency while preserving the model accuracy.