A Least Squares Approach to Linear Discriminant Analysis
Richard M. Crownover · SIAM Journal on Scientific and Statistical Computing · 1991
A new approach to linear discriminant analysis that puts the burden of the computational load on solving a linear least squares problem rather than on solving a generalized eigensystem is presented. The problem being solved is that of reducing the dimension of a vector space in which several classes of objects are to have the best possible separation after the dimension reduction. Two similar algorithms using the least squares approach are developed, one based on the singular value decomposition and the other on orthogonal triangularization (QR factorization). The version using the singular value decomposition is more stable, especially in the nearly rank deficient case. The orthogonal triangularization version is faster, and is the preferred method if updating is anticipated.