Neural network information criterion for the optimal number of hidden units
Takashi Onoda · 2002
This paper presents a statistical approach to solve the problem of model selection, or determine the number of hidden units for artificial neural networks. The authors' approach analyzes the relation between the learning error which is a measure of the convergence and the general generalization error which is a measure of the quality of approximation by a statistical discrepancy and a discrepancy accompanied by the learning algorithm. The author makes clear the relation between the learning error and the generalization error by analyzing the two discrepancies statistically. Moreover, the author derives a new information criterion based on a given learning set by this relation. The author calls this information criterion a "neural network information criterion: NNIC". The author can construct the optimal architecture of multi-layered neural networks for given learning examples by using this criterion. This paper shows that this criterion is effective by a simple simulation which compares NNIC with NIC. Finally, the author points out that NNIC is the generalized form of some information criteria.