ON THE ASYMPTOTIC LINEAR CONVERGENCE OF GRADIENT DESCENT FOR NON-SYMMETRIC MATRIX COMPLETION
Trung Dung Vu, Raviv Raich · 2023
This paper studies a factorization-based gradient descent approach for non-symmetric matrix completion. We introduce an objective that includes an orthogonality regularization for one of the factors. Additionally, we introduce a scaling term to ensure that the two factors are of equal magnitude to improve the convergence speed. For the proposed objective, we analyze the exact linear convergence rate of gradient descent via the asymptotically linear update equation for the error matrix. Our proposed result is the first closed-form expression of the exact linear rate. To illustrate the correctness and tightness of our analysis, we compare the empirical convergence rate against the analytical rate. Additional numerical experiments are done to verify the efficacy of the scaling approach.