Storage Capacity of a Multilayer Neural Network with Binary Weights
Eli Barkai, Ido Kanter · Europhysics Letters (EPL) · 1991
Statistical mechanics is applied to estimate the maximal capacity per weight (α c ) of a two-layer feed-forward network with binary weights, functioning as a parity machine of the hidden units. For K ⩾ 2 hidden units, the maximal theoretical capacity is achieved, α c = 1, and the average overlap between different solutions is zero. These results agree with the simulations. At finite temperature one-step replica symmetry breaking solution is found, which appears to be exact.