The importance of the activation function in NeuroEvolution with FS-NEAT and FD-NEAT
Evgenia Papavasileiou, Bart M. P. Jansen · 2017
The majority of existing NeuroEvolutionary algorithms optimize the connectivity and the topology of the nodes of Artificial Neural Networks (ANNs). However, the architecture of an ANN is also defined on the nodes' activation functions which play a significant role in the network's performance. Feature Selective NeuroEvolution of Augmenting Topologies (FS-NEAT) and Feature Deselective NeuroEvolution of Augmenting Topologies (FD-NEAT) are two methods that optimize the topology and the weights of ANNs, while simultaneously performing feature selection. Since no study exists where FD-NEAT and FS-NEAT are employed on different activation functions, in this study we investigate the importance of the activation function in FS-NEAT and FD-NEAT and its influence in terms of accuracy, number of generations, ability of performing feature selection and size of the evolved networks. The different settings of activation functions are compared using Kruskal-Wallis hypothesis tests with Bonferroni correction (p<;0.01), while FD-NEAT and FS-NEAT are compared using Wilcoxon rank sum hypothesis tests (p<;0.01). The results show that employing hyperbolic tangent in the hidden layer and Gaussian functions in the output layer delivers significantly more efficient algorithms that are faster, more accurate and able to evolve significantly smaller ANNs with better feature selection ability.