Compression-based methods for nonparametric density estimation, on-line prediction, regression and classification for time series
Boris Ya. Ryabko · 2008
We address the problem of nonparametric estimation of characteristics for stationary and ergodic time series. We consider finite-alphabet time series and the real-valued ones and the following problems: estimation of the (limiting) probability P(u0hellipus) for every s and each sequence u0hellip usof letters from the process alphabet (or estimation of the density p(x0,hellip, xs) for real-valued time series), so-called on-line prediction, where the conditional probability P(xt+1/x1x2hellipxt) (or the conditional density p(xt+1/x1x2hellipxt)) should be estimated (in the case where x1x2hellip xtis known), regression and classification (or so-called problems with side information). We show that any universal code (or a universal data compressor) can be used as a basis for constructing asymptotically optimal methods for the above problems.