Optimizing the number of hidden nodes of a feedforward artificial neural network
Lizelle Fletcher, Vladimir Y. Katkovnik, Francois E. Steffens, Andries Petrus Engelbrecht · 2002
The number of hidden nodes is a crucial parameter of a feedforward artificial neural network. A neural network with too many nodes may overfit the data, causing poor generalization on data not used for training, while too few hidden units underfit the model, and is not sufficiently accurate. The mean square error between the estimated network and a target function has a minimum with respect to the number of nodes in the hidden layer, and is used to measure the accuracy. In this paper an algorithm is developed which optimizes the number of hidden nodes by minimizing the mean square error over noisy training data. The algorithm combines training sessions with statistical analyses and experimental design to generate new sessions. Simulations show that the developed algorithm requires fewer sessions to establish the optimal number of hidden nodes than using the straightforward way of eliminating nodes successively one by one.