Extending FISTA to Riemannian Optimization for Sparse PCA
Wen Tao Huang, Kexiang Wei · arXiv (Cornell University) · 2019
Sparse PCA, an important variant of PCA, attempts to find sparse loading vectors when conducting dimension reduction. This paper considers the Riemannian optimization problem related to the ScoTLASS model for the sparse PCA which can impose orthogonality and sparsity simultaneously. We extend FISTA from the Euclidean space to the Riemannian manifold to solve this problem, leading to the accelerated Riemannian proximal gradient method. Since the optimization problem is essentially non-convex, a safeguard strategy is introduced in the algorithm. Moreover, a diagonal weighting strategy is also proposed which can further improve the computational efficiency of the Riemannian proximal methods. Numerical evaluations establish the computational advantages of the proposed methods over the existing proximal gradient methods on manifold. Convergence of the methods to stationary point has also been rigorously justified.