e-Entropy and the Complexity of Feedforward Neural Networks
Robert C. Williamson · Neural Information Processing Systems · 1990
We develop a new feedforward neural network representation of Lipschitz functions from [0, ρ]n into [0, 1] based on the level sets of the function. We show that nρL/2er + 1/√2er + (1+n/√2)(ρL/4er)n is an upper bound on the number of nodes needed to represent f to within uniform error er, where L is the Lipschitz constant. We also show that the number of bits needed to represent the weights in the network in order to achieve this approximation is given by O(n2ρL/√2 4ner (ρL/er)n). We compare this bound with the e-entropy of the functional class under consideration.