Reliability of Replica Symmetry for the Generalization Problem of a Toy Multilayer Neural Network
Andreas Engel, L Reimers · Europhysics Letters (EPL) · 1994
The generalization ability of a spherical perceptron with non-monotonous activation function serving as a toy model for multilayer networks is investigated. With increasing size of the training set there is in replica symmetry a discontinuous transition from a poorly to a well-generalizing phase. The replica-symmetric saddle-point is tested for its local and global stability. The results indicate that for multilayer nets the equilibrium properties of the generalization problem are correctly described in replica symmetry whereas the study of metastable states necessitates replica symmetry breaking.