L-DQN: An Asynchronous Limited-Memory Distributed Quasi-Newton Method
Bugra Can, Saeed Soori, Maryam Mehri Dehnavi, Mert Gürbüzbalaban · 2021 60th IEEE Conference on Decision and Control (CDC) · 2021
This work proposes a distributed algorithm for solving empirical risk minimization problems, called L-DQN, under the master/worker communication model. L-DQN is a distributed limited-memory quasi-Newton method that supports asynchronous computations among the worker nodes. Our method is efficient both in terms of storage and communication costs, i.e., in every iteration, the master node and workers communicate vectors of size O(d), where d is the dimension of the decision variable, and the amount of memory required on each node is O(md), where m is an adjustable parameter. To our knowledge, this is the first distributed quasi-Newton method with provable global linear convergence guarantees in the asynchronous setting where delays between nodes are present. Numerical experiments are provided to illustrate the theory and the practical performance of our method.