An empirical study of the test error versus training error in artificial neural networks
Fernando Morgado‐Dias, Ana Antunes · International Conference on Information and Automation · 2008
This paper reports an empirical study of the behavior of the test and training errors in different systems. Frequently the test error of Artificial Neural Networks is presented with a monotonic decreasing behavior as a function of the iteration number, while the training error also continuously decreases. The present paper shows examples where such behavior does not hold, with data collected from systems where it is corrupted by either noise or actuation delay. This shows that selecting the best model is not a simple question and points to automatic procedures for the selection of models as the best solution to optimize their capacity, either with the Regularization or Early Stopping techniques.