Growth and harvest induce essential dynamics in neural networks
Ilona Markovna Kulikovskikh, Tarzan Legović · Proceedings of the Genetic and Evolutionary Computation Conference Companion · 2021
Training neural networks with faster gradient methods brings them to the edge of stability, proximity to which improves their generalization capability. However, it is not clear how to stably approach the edge. We propose a new activation function to model inner processes inside neurons with single-species population dynamics. The function induces essential dynamics in neural networks with a growth and harvest rate to improve their generalization capability.