Assessing generalization of feedforward neural networks
M. Turmon, Terrence L. Fine · 2002
Neural networks have been used to tackle what might be termed 'empirical regression' problems. Given independent samples of input/output pairs (x/sub i/,y/sub i/), we wish to estimate f(x)=E[Y|X=x]. The approach taken is to choose an approximating class of networks N={/spl eta/(x;w)}w/spl isin/W and within that class, by an often complex procedure, choose an approximating network /spl eta/(/spl middot/;w*). The distance (in mean squared error) of this network from f can be separated into two terms: one for approximation or bias-choosing N large enough so that some /spl eta/(/spl middot/;w/sup 0/), say, models f well-and one for estimation or variance-how well the chosen /spl eta/(/spl middot/;w*) performs relative to /spl eta/(/spl middot/;w/sup 0/). We address the latter term.