UNIFORM CONVERGENCE BOUNDS FOR LEARNING FROM EXAMPLES
Alexander Engel · Modern Physics Letters B · 1994
Basic notions of learning from examples in feed-forward neural networks are reviewed with special emphasis on the relation between the different approaches. First classical results of mathematical statistics on uniform bounds for the convergence of the learning error to the generalization error are discussed from a physical point of view. Recent work of statistical mechanics on the generalization ability of large networks of formal neurons is shown to reproduce and extend these results. In particular for simple architectures the tightness of the convergence bounds as well as the relation between the typical and the worst case performance can be determined in the thermodynamic limit. Several interesting questions remain open.