STORAGE CAPACITY BOUNDS IN MULTILAYER NEURAL NETWORKS
Andreas Wendemuth · International Journal of Neural Systems · 1994
General analytic expressions are given for the lower and upper storage capacity bounds of multilayer neural networks which have variable weights between input layer and first hidden layer, and a fixed output function implemented between first hidden and output layer. The special cases of committee and parity machines as well as the limiting cases of networks with minimum and maximum storage capacities are discussed. The results are compared with replica calculations and simulations. An explanation is given as to why the latter have a storage capacity just slightly above the lower limit and how this can be improved.