Simple iteration-optimal distributed optimization
Konstantinos I. Tsianos, Michael Rabbat · 2013
We propose a consensus-based distributed optimization algo-rithm for minimizing separable convex objectives. Each node only knows one component of the objective function, and so the nodes must coordinate in order to find a global minimizer. The proposed algorithm has an error rate which is no more than O(1/√T) after T iterations, matching the best possible rate. To achieve this, the algorithm requires multiple rounds of consensus per iteration, where the number of consensus rounds depends on the structure of the underlying communi-cation topology through the spectral gap. Consequently, the amount of computation required by the proposed approach is less that of distributed optimization methods in the literature, while the total amount of communication is not increased. Index Terms — Distributed optimization, consensus 1.