Optimization Models for Feature Selection of Decomposed Nearest Neighbor
Cao Xiao, Wanpracha Art Chaovalitwongse · IEEE Transactions on Systems Man and Cybernetics Systems · 2015
The traditional k-nearest neighbor (KNN) method classifies an object by the majority vote of its neighbors, and only one parameter k is to be optimized. We propose a decomposed version of the KKN (DKNN) method, which classifies an unseen object based on the distances to the centroids of the KNNs. DKNN has a training process to learn the local-optimafree distance metric by solving a convex optimization problem. The optimization problem not only learns a metric that minimizes classification errors and maximizes the margin between intraclass and interclass distances. In addition, it also selects important features and removes irrelevant features when L1 regularization is incorporated in the optimization model. The feature selection component is extremely useful for high-dimensional classification problems. Tested on public datasets, the proposed DKNN is competitive and often outperforms traditional machine learning algorithms.