Study on two-stage fractional order gradient descend method
Zhiguang Zhu, Ang Li, Yong Wang · 2021
In this paper, a new kind of gradient descent method with global search capability is proposed. This method is superior to the conventional gradient method in two aspects. On the one hand, the particle swarm optimization algorithm is used to obtain the initial value near the global optimal solution. On the other hand, the fractional order gradient is used instead of the first order in the iteration. For the characteristics of the fractional calculus, the method can quickly escape the saddle point to accelerate the convergence process. We test the efficiency of the algorithm on a multi extremum function and a medium scale convolution neural network, and compare it with other conventional algorithms. Finally, conclusions and some suggestions for future work are given.