Concave penalized linear discriminant analysis on high dimensions
Sunghoon Kwon, Hyebin Kim, Dongha Kim, Sangin Lee · Communications for Statistical Applications and Methods · 2024
The sparse linear discriminant analysis can be incorporated into the penalized linear regression framework, but most studies have been limited to specific convex penalties, including the least absolute selection and shrinkage operator and its variants.Within this framework, concave penalties can serve as natural counterparts of the convex penalties.Implementing the concave penalized direction vector of discrimination appears to be straightforward, but developing its theoretical properties remains challenging.In this paper, we explore a class of concave penalties that covers the smoothly clipped absolute deviation and minimax concave penalties as examples.We prove that employing concave penalties guarantees an oracle property uniformly within this penalty class, even for high-dimensional samples.Here, the oracle property implies that an ideal direction vector of discrimination can be exactly recovered through concave penalized least squares estimation.Numerical studies confirm that the theoretical results hold with finite samples.