An Information Theoretic Linear Discriminant Analysis Method
Haihong Zhang, Cuntai Guan, Kai Keng Ang · 2010
We propose a novel linear discriminant analysis method and demonstrate its superiority over existing linear methods. Based on information theory, we introduce a non-parametric estimate of mutual information with variable kernel bandwidth. Furthermore, we derive a gradient-based optimization algorithm for learning the optimal linear reduction vectors which maximizes the mutual information estimate. We evaluate the proposed method by running cross-validation on 2 data sets from the UCI repository, together with linear and nonlinear SVMs as classifiers. The result attests to the superority of the method over conventional LDA and its variant, aPAC.