96 Replica Symmetry Breaking and the Kuhn-Tucker Cavity Method in simple and multilayer Perceptrons ∗
Franz Gerl, U. Kreyß, D- Gottingen · 2007
Within a Kuhn-Tucker cavity method introduced in a former paper, we study optimal stability learning for situations, where in the replica formalism the replica symmetry can be broken, namely (i) the case of a simple perceptron above the critical loading, and (ii) the case of two-layer AND-perceptrons. We find that the deviation of our cavity solution from the replica symmetric one is a clear indication of the necessity of replica breaking in both cases. However, in case (i) our cavity method seems to overestimate considerably the loading, which is necessary to minimize the errors. 1 Introduction In a recent paper, [1], we have treated the learning problem for general Hopfield perceptrons with Q- and Q 0 state Potts model input and output neurons using a general method based on a cavity formalism. In based on the PhD thesis of F. Gerl, Regensburg 1994 1 this method, the Kuhn-Tucker conditions, which lead to optimal stability in AdaTron type learning processes, have been b...