Lenet-5 Convolution Neural Network with Mish Activation Function and Fixed Memory Step Gradient Descent Method
Zhihao Zhang, Zan Yang, Yuan Sun, Yang-Fan Wu, Yidan Yedda Xing · 2019
Convolutional neural network is the most important algorithm in the field of deep learning. The traditional convolution neural network usually uses Sigmoid or Relu as the activation function, but the two sides of Sigmoid are saturated, and Relu has a dead zone, which is very easy to cause gradient disappearance and gradient explosion. In this paper, Mish activation function is introduced into LENET-5 convolutional neural network, which overcomes the shortcomings of traditional activation function. At the same time, the fixed memory step gradient descent method is used to replace the gradient descent method of the optimization part, which improves the global convergence of the algorithm.