Accelerated Zeroth-Order Algorithm for Stochastic Distributed Non-Convex Optimization
Shengjun Zhang, Colleen P. Bailey · 2022 American Control Conference (ACC) · 2022
This paper investigates how to accelerate the convergence of distributed optimization algorithms on non-convex problems with only zeroth-order information available. We propose a zeroth-order (ZO) distributed primal-dual stochastic coordinate algorithm equipped with "powerball" method to accelerate the convergence. We establish the convergence result of the considered algorithm for general non-convex cost functions. We verify the proposed algorithm through the benchmark example of generating adversarial examples from black-box DNNs in order to compare with the existing state-of-the-art centralized and distributed ZO algorithms. The numerical results demonstrate a faster convergence rate for the proposed algorithm and match the theoretical analysis.