An empirical study of L2-Boost with Echo State Networks
Sebastián Basterrech · 2013
At the beginning of the 2000s was introduced the Echo State Network model (ESN). The model has been successfully used in temporal learning tasks. In spite of its success in practical applications, the model can present some stability problems when the parameters are not well initialized. The stability of the model is associated with the spectrum of the weight matrix. To compute the spectra is an expensive tasks when the network is large. Below, the initialization of the network parameters can depend of the benchmark problem. In this paper, we investigate the performance of the L2-Boost procedure, one specific boosting technique, in time-series problems. We use an ensemble of ESNs which are randomly initialized without control of the spectral radius norm as weak predictors of the L2procedure. Therefore, the procedure consists only in a random initialization of an ensemble of ESNs following of the L2-Boost steps.We evaluate this procedure on 5 widely used time-series benchmarks. Furthermore, we compare this procedure with a baseline approach which consists of averaging the prediction of an ensemble of ESNs with different initial network settings.