On alpha-mean of Kullback-Leibler divergences for subspace selection
Si-Bao Chen, Haixian Wang, Xingyi Zhang, Bin Luo · 2011
Fisher's linear discriminant analysis (FLDA) is one of the most well-known linear subspace selection methods. However, FLDA suffers from the class separation problem. The projection to a subspace tends to merge close class pairs. Recent results show that maximizing the geometric mean or harmonic mean of Kullback-Leibler (KL) divergences of class pairs can significantly reduce this problem. In this paper, to further reduce the class separation problem, the alpha-mean of divergences as a framework is proposed for subspace selection, with arithmetic mean, geometric mean and harmonic mean as special cases. We named it the maximization of the alpha-mean of all pairs of KL divergences (MAMD) criterion. A quasi-Newton method is applied to solve the optimization problem. Experiments on synthetic data and two datasets in UCI machine learning repository show the validity of MAMD.