Training data selection by detecting predictability in non-stationary time series by a surrogate-cumulant based approach
Gustavo Deco, Bernd Schürmann · 2002
We introduce a nonparametric cumulant based statistical approach for detecting linear and nonlinear statistical dependences in nonstationary time series. The statistical dependence is detected by measuring the predictability which tests the null hypothesis of statistical independence, expressed in Fourier-space, by the surrogate method. Therefore, the predictability is defined as a higher-order cumulant based significance discriminating between the original data and a set of scrambled surrogate data which correspond to the null hypothesis of a noncausal relationship between past and present. Information about the predictability can be used for example to select regions where a temporal structure is visible in order to select data for training a neural network for prediction. The regions where only a noisy behavior is observed are therefore ignored, avoiding in this fashion the learning of irrelevant noise which normally spoils the generalization characteristics of the neural network. We present an example of nonstationarity given by the chaotic time series of Henon (1976) perturbed with linearly increasing additive Gaussian noise. Nonlinear structures are tested in financial time series, like the Dollar-DM Tick exchange rate.