Error vs. Rejection Curve for the Perceptron
Juan M. R. Parrondo, C. van den Broeck · Europhysics Letters (EPL) · 1993
We calculate the generalization error ε for a "student" perceptron J , trained by a teacher perception T , on input patterns S that form a fixed angle arccos ( J · S ) with the student. We show that the error is reduced from a power law to an exponentially fast decay by rejecting input patterns that lie within a given neighbourhood of the decision boundary J · S = 0. On the other hand, the error vs. rejection curve ε ρ , where ρ is the fraction of rejected patterns, is shown to be independent of the training scheme that is employed to construct the student perceptron. We give a simple argument indicating that the small-ρ behavior observed for the perceptron ε ρ = ε 0 + ρ(ε 0 − 1/2) has a much wider range of validity.