Nonlinear Activation Functions for Artificial Neural Networks Realized in Hardware
Zofia Długosz, Rafal Tomasz Dlugosz · 2018
The paper presents a hardware efficient implementation of selected nonlinear activation functions of the neuron for the application in various artificial neural networks, including wavelet neural network (WNNs). Similar solutions may also be used in fuzzy neural networks. A software implementation of the activation function is relatively simple, however in hardware the realization is more complex. For this reason, we performed investigations, in which the training process was completed with simplified activation function. The comparison with the results obtained for an ideal function have shown that such a simplification is acceptable. The realized WNN has been successfully verified with selected signals composed of trigonometric functions, accompanied by the Gaussian noise.