Off-line learning from clustered input examples
Carmela Marangi, Sara A. Solla, Michael L. Biehl, Peter Riegler · 1996
We analyze the generalization ability of a simple perceptron acting on a structured input distribution for the simple case of two clusters of input data and a linearly separable rule. The generalization ability computed for three learning scenarios: maximal stability, Gibbs, and optimal learning, is found to improve with the separation between the clusters, and is bounded from below by the result for the unstructured case, recovered as the separation between clusters vanishes. The asymptotic behavior for large training sets is the same for structured and unstructured input distributions. For small training sets, the generalization error of the maximally stable perceptron exhibits a nonmonotonic dependence on the number of examples for certain values of the model parameters. 1. Introduction The problem of supervised learning is usually formulated 1\\Gamma4 as that of a student network architecture being trained from examples in order to implement a target input-output relation. The ...