Algorithm Research on Improving Activation Function of Convolutional Neural Networks
Yanhua Guo, Lei Sun, Zhihong Zhang, Hong Gu He · 2019
Aiming at the slow convergence of the activation function of Sigmoid, Tanh, ReLu and Softplus as the model and the non-convergence caused by gradient dispersion, this paper proposes an algorithm to improve the activation function of convolutional neural network. Using R-SReLU as the activation function of the neural network, the convergence speed of various excitation functions to the network and the accuracy of image recognition are analyzed. The experimental data shows that the improved activation function R-SReLU not only has a fast convergence speed, but also has a small error rate, and can improve the accuracy of classification more effectively. The maximum recognition accuracy reaches 88.03%.