A Stochastic Momentum Accelerated Quasi-Newton Method for Neural Networks (Student Abstract)
S. Indrapriyadarsini, Shahrzad Mahboubi, Hiroshi Ninomiya, Takeshi Kamio, Hideki Asai · Proceedings of the AAAI Conference on Artificial Intelligence · 2022
Incorporating curvature information in stochastic methods has been a challenging task. This paper proposes a momentum accelerated BFGS quasi-Newton method in both its full and limited memory forms, for solving stochastic large scale non-convex optimization problems in neural networks (NN).