Optimizing Variable Selection and $k$ in the $k$-NN Classifier with Precision Objective
Carrie K. Y. Lin · 2023
This paper focuses on maximizing the precision in binary classification problems using the$k$-Nearest Neighbour (k-NN) algorithm by simultaneously selecting the variables and neighbourhood size$(\boldsymbol{k})$. The inputs to$k$-NN include a set of variables, the neighbourhood size and the distance metric usually selected based on data characteristics. The first two are typically decided sequentially in many studies. The current simultaneous optimization problem is formulated by a mixed-integer linear fractional program and solved by parametric algorithm. The squared Euclidean distance metric is used but the model can be adapted for other distance metrics. The methodology is tested on ten publicly available datasets. Results showed that using at least half to all variables with appropriate$\boldsymbol{k}$value can achieve better or equally good precision. An effective set of variables jointly determined with neighbourhood size can facilitate$k$-NN to perform classification with high precision.