Optimal Conjugate Gradient Algorithm for Generalization of Linear Discriminant Analysis Based on L1 Norm
Kanishka Tyagi, Nojun Kwak, MICHAEL T. MANRY · 2014
This paper analyzes a linear discriminant subspace technique from an L-1 point of view. We propose an efficient and optimal algorithm that addresses several major issues with prior work based on, not only the L-1 based LDA algorithm but also its L-2 counterpart. This includes algorithm implementation, effect of outliers and optimality of parameters used. The key idea is to use conjugate gradient to optimize the L-1 cost function and to find an optimal learning factor during the update of the weight vector in the subspace. Experimental results on UCI datasets reveal that the present method is a significant improvement over the previous work. Mathematical treatment for the proposed algorithm and calculations for learning factor are the main subject of this paper.