Estimation of size of hidden layer on basis of bound of generalization error
B. Igelnik, Yoh‐Han Pao · 2002
For the class of functions with regularity expressed in terms of its Fourier transform, a simple algorithm is suggested for estimation of the size of single hidden layer feedforward neural network for approximating a function from the given class. We applied this algorithm to the architecture of the single hidden layer neural network, namely the random version of the functional-link net (FLN). Discussion of generalization capabilities of the FLN is presented. The algorithm is based on the bound, derived by Barren (1993), for the mean integrated squared error between the network estimate and the target function. It determines the estimate of the minimal number of nodes sufficient to guarantee that the error be less than a prescribed level, if this level of error is achievable for the given training data. In the opposite case the algorithm finds estimate of the size of the network which guarantees that the error be within a small interval containing the minimal achievable value of the error. An iterative procedure of the algorithm allows one to avoid the need for involved calculations of the constants contained in the bound.