MLP Can Provably Generalize Much Better than VC-bounds Indicate
Adam P. Kowalczyk, Herman Ferrá · neural information processing systems · 1996
Results of a study of the worst case learning curves for a particular class of probability distribution on input space to MLP with hard threshold hidden units are presented. It is shown in particular, that in the thermodynamic limit for scaling by the number of connections to the first hidden layer, although the true learning curve behaves as ≈ α-1 for α ≈ 1, its VC-dimension based bound is trivial (≡ 1) and its VC-entropy bound is trivial for α ≤ 6.2. It is also shown that bounds following the true learning curve can be derived from a formalism based on the density of error patterns.