Analysis of Features of Different Activation Functions
Kewen Liu · 2021 2nd International Conference on Computing and Data Science (CDS) · 2021
Inspired by the mechanisms of biological neural cells, artificial neural network (ANN) has a wide application over many areas for its predictive performances of data analysis. Devised as the mathematic “gate”, activation function plays a key role in ANN to determine the degree of activation of each neuron unit. The choice of activation function usually has a strong influence on optimization and model accuracy. In this paper, several mathematical properties of activation functions are analyzed through the studies of relating literatures. It's concluded that the selection of activation function is not an achievement at one stroke but what should take considerations from different aspects.