Number of Hidden Units in Function Approximation by Neural Networks Using ^|^epsilon;-Entropy
Yoshihide MORI, Michifumi Yoshioka, S. Ōmatu · Transactions of the Society of Instrument and Control Engineers · 1999
An important problem when one develops the neural networks is to determine a suitable size of a neural network. In this paper, we will consider to find the number of hidden units thoretically, since the number of units in input and output layers has not been determined so much from thoretical viewpoint. We will introduce Kolmogorov's ε-entropy and ε-capacity in neural networks, and calculate ε-entropy and ε-capacity of neural networks to approximate Lipschitz continuous functions. Then by numerical simulations we will show the effectiveness of the proposed algolithm to find the number of units.