Backpropagation in time series analysis
Gerson Lachtermacher · 1993
As pointed out by Weigend et al. (1990, 1991), two of the major constriants to the use of backpropagation neural networks as a practical forecasting tool, are the number of training patterns required and the long modelling time. In this thesis we propose and test the Hybrid methodology that reduces the data requirement and decreases the modelling time. The general idea is to use the validation procedure (Weigend, 1990). However, in order to reduce the data requirement we use the Box-Jenkins calibration procedure (Hipel & McLeod, 1977, 1993), to identify the 'lag components' of the time series that we want to model. The calibrated ARMA/ARIMA model suggests the number of input units to be used in the network structure. This process reduces the size of the network and consequently the data required to train the network. Furthermore, the use of the Box-Cox transformation, when suggested by the calibrated ARMA/ARIMA model, is demonstrated to be an important feature in the neural network calibration process. It should be noted that this type of transformation dramatically reduces the number of cycles of the neural network training process. Therefore it collaborates in the reduction of the neural network modelling time. Moreover, in order to avoid the need for data as a validation set, as done by Weigend (1991), the Hybrid methodology uses a synthetic time series generated by the calibrated ARMA/ARIMA model. This series is used to determine the stopping point where the training should be interrupted in order to avoid the overfitting problem. The utilization of the synthetic data as the validation set, reduces the amount of data needed for the neural network modelling process. In almost all of the models, the synthetic series mimic very closely the original series, which suggests that these series can be used as a very good approximation of the original series for validation purposes. This methodology has been tested in basically two types (stationary and nonstationary) of time series. Four annual riverflow time series have been used to represent the stationary series while four annual electricity consumption time series represented the nonstationary series. Furthermore a brief study using a cyclical time series was also performed. In the stationary models the Hybrid methodology had the best overall performance over all methods tested. However, the differences in the prediction performance in both types of forecast (one step and multi-step) were not significant enough to justify the additional work needed to calibrate the neural network model. In the nonstationary models the Hybrid methodology outperformed the corresponding ARIMA model in three of the four models tested and performed almost as well as the ARIMA in the other model. In addition the differences in the prediction performance were large enough to justify the additional work to calibrate neural network model. Furthermore, in two of the four models the neural networks outperformed the corresponding ARIMA model in a ratio of at least 3:1, in the multi-step prediction. In the case of the cyclical time series the Hybrid methodology performed worse than the traditional methods (TAR and ARMA) and two other neural network methodologies (weight-elimination and soft weight-sharing). This fact suggests that further research should be done to improve the performance of the Hybrid methodology in the case of cyclical time series, such as an initial phase of deseasonalitions or moving average calculations.