A Stochastic Consensus Method for Nonconvex Optimization on the Stiefel Manifold
Jeongho Kim, Myeongju Kang, Dohyun Kim, Seung‐Yeal Ha, Insoon Yang · 2020
In this paper, we propose a consensus-based algorithm for nonconvex optimization on the Stiefel manifold. For a given objective function on the Stiefel manifold, we construct a stochastic interacting particle system for sample points so that all the sample points are expected to asymptotically converge to a single point, which is close enough to a global minimizer. We show the global existence and uniqueness of solutions to our stochastic differential equation (SDE) model for consensus. A predictor-corrector type numerical scheme is then proposed for implementing the SDE model with the guarantee that each sample point stays on the Stiefel manifold. A salient feature of our algorithm is that it is gradient-free, thereby applicable to a wide range of problems. The results of our numerical experiments demonstrate that the proposed method can successfully find a global minimizer even when the objective function is nonconvex.