Convergence of Gradient Method With Momentum for Two-Layer Feedforward Neural Networks
N. Zhang, Wei Han Wu, Guojun Zheng · IEEE Transactions on Neural Networks · 2006
A gradient method with momentum for two-layer feedforward neural networks is considered. The learning rate is set to be a constant and the momentum factor an adaptive variable. Both the weak and strong convergence results are proved, as well as the convergence rates for the error function and for the weight. Compared to the existing convergence results, our results are more general since we do not require the error function to be quadratic.