Learning dynamics: a replica approach
Ferenc Pázmándi · Journal of Physics A Mathematical and General · 1992
A continuous-time approximation for the learning process in single-layer perceptrons is presented. Both statics and dynamics are treated by the replica method within the framework of the replica symmetric theory. The Adaline rule extended by a decay factor is investigated in detail, especially for linearly-separable problems. It is shown that the poor generalization ability of this rule near alpha =1, the so called overfitting, can be cured by an appropriate decay factor or by an appropriate training time.