An Ensemble Approach to Instance-Based Regression Using Stretched Neighborhoods
Vahid Jalali, David B. Leake · The Florida AI Research Society · 2013
Instance-based methods generate solutions from local estimates, based on prior solutions which fall within a neighborhood of the input query. Such approaches have proven useful, but maximizing performance depends on both the criteria for neighborhood selection and the method used to generate a solution value from the values of prior instances in that neighborhood. This paper proposes a new approach to addressing both problems for regression tasks. In this approach, values for input queries are generated by an ensemble of solutions of local linear regression models built for a collection of “stretched” neighborhoods of the current problem. Each neighborhood is generated by relaxing a different dimension of the problem space. The rationale for this relaxation is to enable major change trends along that dimension to have increased influence on the corresponding model. The performance of this approach for two candidate relaxation approaches, gradient-based and based on fixed profiles, is compared to the baseline of using a radius-based spherical neighborhood in n-dimensional space. Results in four test domains show up to 15 percent improvement over baselines of k-NN and using local radius-based spherical neighborhood for training a linear regression model. They suggest that the new methods could be particularly useful in domains for which the set of prior instances is sparse.