Comment on “The use of artificial neural networks for the prediction of water quality parameters” by H. R. Maier and G. C. Dandy
Vincent Fortin, Taha B. M. J. Ouarda, Bernard Bobée · Water Resources Research · 1997
Nonparametric approaches to prediction and forecasting of complex physical systems, such as artificial neural networks (ANNs), are set to spread with modern computing possibilities.As Maier and Dandy [1996] correctly point out, data-driven models make prediction and forecasting possible under less stringent hypotheses.This paper is highly appreciated for introducing ANNs to the water resources community and presenting a practical application.However, as many readers may be unfamiliar with this type of model, we would like to clarify links that exist between ANNs and autoregressive-moving average (ARMA) models, which the authors did not emphasize.We would also like to suggest the use of a different network configuration, which may prove more appropriate for time series forecasting.To discuss the links that exist between ANNs and ARMA models, a common vocabulary is needed.ANNs, which were originally designed to solve artificial intelligence (AT) problems such as speech and hand writing recognition, are often described using AT terminology.While this may have been appropriate when the purpose of ANNs was to loosely model the human brain, it is now a major source of confusion.For example, the authors name "training" or "learning" the process by which the weights of an ANN are adjusted so as to minimize mean squared error (section 2.2.3).However, in fact, this is absolutely equivalent to the calibration process of any stochastic model: training is equivalent to "calibrating."The algorithm most often used for calibration of an ANN, called "backpropagation" by the AT community (see section 3 for instance) is nothing else but the well-known steepest-descent method of optimization [Cauchy, 1847].Backpropagation also refers in some of the literature (and in this paper) to the type of feed forward network used by the authors, but this terminology is misleading as other equally valid algorithms may be used for calibration, including the conjugate gradient method [Fletcher and Reeves, 1964], simulated annealing [Aarts and Korst, 1989], genetic algorithms [Holland, 1992], and evolutionary programming [Fogel et al., 1989].At this point in their development, ANNs are simply another type of "black-box" model [Weisread, 1994].An ANN needs to be configured and calibrated like any other type of model, and using similar algorithms, it does not, in any way, "learn" a relationship from the data, unless we define as learning the process of fitting a model to observed data.Machine learning is best defined by Mogili and Sunol [1993, p. 756] as "a process in which a computer program improves its performance, acquires knowledge and solves new problems in a specified domain".These authors also correctly state that ANNs "... are unsuitable for knowledge acquisition purposes