Nonlinear Activation in Deep Residual Networks
Qi‐Jun Zhang, Songling Fu, Dan Li · 2020
In deep neural network, the Deep Residual Network (ResNets) is very representative, which greatly improves the depth of the network and skillfully avoids the problem of gradient disappearance in the deep network. The network structure has been improved in convergence speed and accuracy. Some papers have analyzed the residual structure, including different branch structure, different function positions, forward propagation process, etc., and proposed preactivation and post-activation. This paper deeply analyzes nonlinear activation in the residual network, put forward the corresponding optimization scheme: New Conv block structure. At the same time this paper also did a lot of experiments, to validate the optimization scheme, the experimental results show that the classification performance of the proposed scheme on CIFAR-10 is improved, and classification accuracy of SlowFast Network on the UCF-101 is greatly improved.