Toward Optimized Federated Learning With Compressed Communications by Rate Adaption
Laizhong Cui, Xiaoxin Su, Yipeng Zhou, Jiangchuan Liu, Shiting Wen · IEEE Transactions on Networking · 2025
It is known that federated learning (FL) incurs heavy communication overhead for model training by exchanging model updates between clients and the parameter server (PS) over the Internet for multiple rounds. Compressing model updates is an effective approach to alleviating communication overhead in FL. Yet the tradeoff between compression and model accuracy in the networked environment remains unclear and, for simplicity, most implementations adopt a fixed compression rate only during the entire learning process. In this paper, we for the first time systematically examine this tradeoff, explicitly quantifying the relation between the compression error, the final model accuracy and the learning rate. Specifically, we factor the compression error of each global iteration into the convergence rate analysis under non-convex loss for both unbiased and biased compression algorithms. We then present an adaptation framework to maximize the final model accuracy by strategically adjusting the compression rate in each iteration. We further discuss key implementation issues of our framework in practical networks with classical compression algorithms. Experiments over the most representative MNIST, CIFAR-10 and CIFAR-100 datasets demonstrate that our solutions effectively shrink network traffic volume while maintain high model accuracy in FL.