Asynchronous Decentralized Consensus ADMM for Distributed Machine Learning
Jiafeng Zhang · 2019
Decentralized consensus optimization algorithms are often applied in peer-to-peer network where the agents communicate with their neighbors and perform local computation. However, in many cases, the synchronous decentralized algorithms suffer from the straggler problem and the consequent communication latency. This paper proposes an asynchronous decentralized consensus alternating direction method of multipliers (ADMM) algorithm that can effectively reduce the communication latency. In the algorithm, each agent can compute and communicate independently under two control conditions (partial barrier, bounded delay), non-blocking communication is adopted to make the communication and computation perform simultaneously, and the step-size of the algorithm is adjusted to accelerate convergence. We test the algorithm on a high-performance computing cluster, experiments on distributed sparse logistic regression show that the asynchronous algorithm achieves faster convergence meanwhile the network waiting time is significantly reduced.