Simultaneous Selection of Features and Metric for Optimal Nearest Neighbor Classification
David A. Johannsen, Edward J. Wegman, Jeffrey L. Solka, Carey E. Priebe Β· Communication in Statistics- Theory and Methods Β· 2004
Given a set of observations in β n along with provided class labels, π, one is often interested in building a classifier that is a mapping from β n β π. One way to do this is using a simple nearest neighbor classifier. Inherent in the use of this classifier is a metric or pseudo-metric that measures the distance between the observations. One typically uses the L 2 metric. We examine the classification benefits of the use of alternative Minkowski p-metrics. We also study the relationship between the selection of the p-metric and the selection of optimal classification features. We compare a simple greedy approach of Minkowski p-metric optimization followed by feature selection, the greedy method, with a simultaneous optimization of the p-metric and feature selection process. We utilize a stochastic optimization methodology to perform the simultaneous optimization.