Generalization in a Large Committee Machine
Holm Schwarze, John Hertz · Europhysics Letters (EPL) · 1992
We study generalization in a committee machine with nonoverlapping receptive fields trained to implement a function of the same structure. Using the replica method, we calculate the generalization error in the limit of a large number of hidden units. For continuous weights the generalization error falls off asymptotically inversely proportional to α, the number of training examples per weight. For binary weights we find a discontinuous transition from poor to perfect generalization followed by a wide region of metastability. Broken replica symmetry is found within this region at low temperatures. The first-order transition occurs at a lower and the metastability limit at a higher value of α than in the simple perceptron.