Evolving a Locally Optimized Instance Based Learner
Ulf Johansson, Rikard König, Lars Niklasson · Borås Academic Digital Archive (University of Borås) · 2008
Standard kNN suffers from two major deficiencies, both related to the parameter k.First of all, it is well-known that the parameter value k is not only extremely important for the performance, but also very hard to estimate beforehand.In addition, the fact that k is a global constant, totally independent of the particular region in which an instance to be classified falls, makes standard kNN quite blunt.In this paper, we introduce a novel instance-based learner, specifically designed to avoid the two drawbacks mentioned above.The suggested technique, named G-kNN, optimizes the number of neighbors to consider for each specific test instance, based on its position in input space; i.e. the algorithm uses several, locally optimized k's, instead of just one global.More specifically, G-kNN uses genetic programming to build decision trees, partitioning the input space in regions, where each leaf node (region) contains a kNN classifier with a locally optimized k.In the experimentation, using 27 datasets from the UCI repository, the basic version of G-kNN is shown to significantly outperform standard kNN, with respect to accuracy.Although not evaluated in this study, it should be noted that the flexibility of genetic programming makes sophisticated extensions, like weighted voting and axes scaling, fairly straightforward.