Asynchronous Stochastic Gradient Descent with Model Quantization
Yubo Du, Keyou You · IFAC-PapersOnLine · 2025
Distributed optimization is pivotal for large-scale machine learning, yet communication costs remain a significant bottleneck. This paper introduces an asynchronous stochastic gradient descent algorithm augmented with a randomized quantization operator to compress model parameters, thereby substantially reducing communication overhead. Our proposed method maintains convergence guarantees under standard regularity assumptions and achieves near-optimal performance in distributed environments. Comprehensive empirical evaluations demonstrate the algorithm’s efficiency and scalability, validating its effectiveness in practical large-scale learning scenarios.