Test error versus training error in artificial neural networks for systems affected by noise
Fernando Morgado‐Dias, Ana Antunes · 2008
This paper reports an empirical study of the behavi or of the test and training errors in different system s. Frequently the test error of Artificial Neural Networks is presented wi th a monotonic decreasing behavior as a function of the iteration number, while the training error also continuously decreases. The pre sent paper shows examples where such behavior does not hold, with data collected from systems where it is corrupted by either noise or actuation delay. This shows that selecting the best model is not a s imple question and points to automatic procedures for the selection of models as the best solution to optimize their capacity, either with th e Regularization or Early Stopping techniques. HIS paper reports an empirical study of the behavio r of the test and training errors in different systems. It is very common in the literature to present the test error of an Artificial Neural Network (ANN) with a monotonic decreasing behavior as a function of the iteration number, while the training error also continuously decrease s. This behavior is, most of the times, illustrated by draw ings instead of simulations or data from a real system. Some exa mples of exceptions can be found in (1) and (2). The present paper shows examples where such behavior does not hold, with data collected from systems whe re it is corrupted by either noise or actuation delay. The behavior of the test error presented points to automatic procedures for the selection of models as the best solution to optimize their capacity, either with the Regulariza tion or Early Stopping techniques. The models presented here were trained using a non- variable pre-established initial set of weights to enable the comparison of the results without the random effect of a variable set of weights.