How limited training data can allow a neural network to outperform an 'optimal' statistical classifier
L.T. Niles, H.F. Silverman, Gary N. Tajchman, Marcia A. Bush · International Conference on Acoustics, Speech, and Signal Processing · 2003
Experiments comparing artificial neural network (ANN), k-nearest-neighbor (KNN), and Bayes' rule with Gaussian distributions and maximum-likelihood estimation (BGM) classifiers were performed. Classifier error rate as a function of training set size was tested for synthetic data drawn from several different probability distributions. In cases where the true distributions were poorly modeled, ANN was significantly better than BGM. In some cases, ANN was also better than KNN. Similar experiments were performed on a voiced/unvoiced speech classification task. ANN had a lower error rate than KNN or BGM for all training set sizes, although BGM approached the ANN error rate as the training set became larger. It is concluded that there are pattern classification tasks in which an ANN is able to make better use of training data to achieve a lower error rate with a particular size training set.>