Convexity, internal representations and the statistical mechanics of neural networks
Manfred Opper, P Kuhlmann, A Mietzner · Europhysics Letters (EPL) · 1997
We present an approach to the statistical mechanics of feedforward neural networks which is based on counting realizable internal representations by utilizing convexity properties of the weight space. For a toy model, our method yields storage capacities based on an annealed approximation, which are in close agreement with one-step replica symmetry-breaking results obtained from a standard approach. For a single-layer perceptron, a combinatorial result for the number of realizable output combinations is recovered and generalized to fixed stabilities.