Phase transitions in simple learning
John Hertz, Anders Krogh, G I Thorbergsson · Journal of Physics A Mathematical and General · 1989
The authors investigate learning in the simplest type of a layered neural network, the one layer perceptron. The learning process is treated as a statistical dynamical problem. Quantities one is interested in include the relaxation time (the learning time) and the capacity and how they depend on noise and constraints on the weights. The relaxation time is calculated as a function of the noise level and the number p of associations to be learned. They consider three different cases for input patterns that are random and uncorrelated. In the first, where the connection weights are constrained to satisfy N -1 Sigma i omega i 2 =S 2 , there is a critical value of p(