Exploring the Impact of Parameters on the Effectiveness of the Neural Network Alternate Incremental Learning Algorithm
Elena S. Abramova, Alexey Orlov, Oleg A. Abramov · 2024
This article investigates the influence of various parameters on the Alternate Incremental Learning Algorithm within the context of neural networks. The primary focus is on three prominent factors: the number of neurons in the hidden layer, the choice of activation function, and the distribution range of weight coefficients. An experimental approach was adopted where the number of hidden layer neurons was varied between 5 and 50, thereby providing insights into how network complexity affects learning capacity. The considered activation functions encompass the sigmoid, rectified linear unit, and hyperbolic tangent, each possessing distinct characteristics that impact network convergence and generalization. Weight coefficients were generated across three ranges: -0.1 to 0.1, -0.5 to 0.5, and -1 to 1, to examine their influence on the network's hypothesis space and learning stability. The study utilized synthetic data analogous to smartphone sensor outputs, targeting the future application of activity recognition. The results summarized in three detailed tables provide a comprehensive analysis of each parameter's impact on the effectiveness of the Alternate Incremental Learning Algorithm.