Nonlinear neural networks near saturation
J. Leo van Hemmen · Physical Review A · 1987
Nonlinear neural networks are studied near saturation, when the number $q$ of stored patterns is proportional to the system size $N$, i.e., $q=\ensuremath{\alpha}N$. The statistical mechanics is obtained for arbitrary nonlinearity. For a wide class of models, including the original Hopfield model and clipped synapses, it is shown that there exists a critical ${\ensuremath{\alpha}}_{c}$ above which the system looses its memory completely. Furthermore, ${\ensuremath{\alpha}}_{c}$ never exceeds ${\ensuremath{\alpha}}_{c}^{\mathrm{hopfield}}$ and is determined by a universal expression. A moderate dilution of the bonds may improve the memory function.