Convergence of Hyperbolic Neural Networks Under Riemannian Stochastic Gradient Descent
Wes Whiting, Bao Wang, Jack X. Xin · Communications on Applied Mathematics and Computation · 2023
Abstract We prove, under mild conditions, the convergence of a Riemannian gradient descent method for a hyperbolic neural network regression model, both in batch gradient descent and stochastic gradient descent. We also discuss a Riemannian version of the Adam algorithm. We show numerical simulations of these algorithms on various benchmarks.