The effect of correlations in neural networks
Andreas Wendemuth, Manfred Opper, Wolfgang Kinzel · Journal of Physics A Mathematical and General · 1993
The effect of correlations in neural networks is investigated by considering biased input and output patterns. Statistical mechanics is applied to study training times and internal potentials of the MINOVER and ADALINE learning algorithms. For the latter, a direct extension to generalization ability is obtained. Comparison with computer simulations shows good agreement with theoretical predictions. With biased patterns, the authors find a decrease in training times and internal potentials for the MINOVER algorithm, which, however, does not lead to faster storage of a given information measure. In ADALINE training, characteristic times undergo a transition from order 1 to order N at any finite bias, for the learning of patterns as well as for the decay of the generalization error. This leads to a rescaling of the gain parameters.