Multiclass linear dimension reduction via a generalized Chernoff bound
Madan Thangavelu, Raviv Raich · 2008
In this paper, we consider the problem of linear dimension reduction (LDR) for multiclass classification. Often, a linear projection in which classes are separable may exist, but is hard to find. In the absence of methods that can find such plane, one may unnecessarily resort to nonlinear dimension reduction (DR). Generalization of two-class separation criteria such as Mahalanobis, Bhattacharya, or Chernoff distance are often done in an ad-hoc fashion. In this paper, we propose two algorithms for multiclass LDR that aim at minimizing upper bounds on the probability of misclassification and are based on generalizations of Chernoff distance for the multiclass problem. We present a numerical study and comparison to state-of-the-art LDR methods on datasets from the UCI machine learning repository. We show that our algorithms result in lower classification error rates compared to techniques of the same class.