Distributed Saddle-Point Problems: Lower Bounds, Optimal and Robust Algorithms
Aleksandr Nikolaevich Beznosikov, Valentin Samokhin, Alexander Vladimirovich Gasnikov · arXiv (Cornell University) · 2020
This paper focuses on the distributed optimization of smooth stochastic saddle-point problems. The first part of the paper is devoted to lower bounds for the cenralized and decentralized distributed methods for smooth (strongly-)convex-(strongly-)concave saddle-point problems as well as the optimal algorithms by which these bounds are achieved. Next, we present a new federated algorithm for saddle-point problems - Extra Step Local SGD. Theoretical analysis of the new method is carried out for (strongly-)convex-(strongly-)concave and non-convex-non-concave problems. In the experimental part of the paper, we show the effectiveness of our method in practice. In particular, we train GANs in a distributed manner.