Adaptive memory based regression methods
Hugues Bersini, Mauro Birattari, Gianluca Bontempi · 2002
The task of approximating a nonlinear mapping using a limited number of observations, asks the data analyst to make several choices involving the set of relevant variables and observations, the learning algorithm, and the validation protocol. In the case of models which are linear in the parameters (e.g. polynomials), statistical theory and economical cross-validation methods provide fast and effective ways to support these choices. However, when pure approximation performance is at stake, a unique linear structure to cover the whole range of data, is often far from optimal. Memory-based methods in contrast are well known to considerably improve the approximation performance, since all the regression analysis is done locally and repeated for each new query. In this paper, we discuss the use of these cross-validation procedures for selecting the features, the neighbors and the polynomial degree for each prediction. The possible automation of these selections on a query basis provides memory-based methods (generally not used in such a flexible way) with a larger degree of adaptivity. Experimental results in time series prediction are presented.