A Generalized-Momentum-Accelerated Hessian-Vector Algorithm for High-Dimensional and Sparse Data
Weiling Li, Xin Luo · 2020
Precisely understanding high-dimensional and sparse (HiDS) user-item interactions is the most important issue in a recommender system. A latent factor analysis (LFA)-based model has proven to be efficient in addressing it, while current models of this kind mostly rely on first-order optimizers. It is vital to implement an LFA-based model able to approach the second order stationary points efficiently for improving its representative learning ability. To do so, this work presents a Generalized-momentum-accelerated Hessian-vector Algorithm (GHA) for HiDS data. Its main idea includes a) adopting the principle of a Hessian-vector-product-based method to avoid operating a Hessian matrix directly, and b) incorporating a generalized momentum method into its parameter learning process for further enhancing its ability in approaching a stationary point. Experimental results on two industrial datasets demonstrate that when compared with state-of-the-art LFA-based models, a GHA-based LFA model achieves gains in accuracy and convergence rate. These positive outcomes also indicate that a generalized momentum method is compatible with algorithms implicitly relying on gradients like a second-order algorithm.