Convergence of Stochastic Gradient Decent Algorithm with Momentum
Jieling Wang, Gang Xie · 2022 IEEE 2nd International Conference on Electronic Technology, Communication and Information (ICETCI) · 2022
This paper aims to discuss the convergence of the stochastic gradient decent algorithm with momentum for feed-forward networks without hidden layers. Assuming that the required sample set is linearly independent, let all training samples be randomly grouped in each round of training, the number of samples in each group is fixed, and the weights are updated in the unit of group. The momentum and learning rate are selected in an adaptive manner, then we prove the convergence results of the learning process.