A Margin Based Feature Extraction Algorithm for the Small Sample Size Problem
Mingyi He · Chinese Journal of Computers · 2007
Feature extraction techniques are widely employed to reduce the dimensionality of data and to enhance the discriminatory information for the classification and recognition tasks.Linear Discriminant Analysis(LDA)is the most popular supervised method for feature extraction,but it often suffers the small sample size problem due to the singularity of the within-class scatter which arises if the number of samples is smaller than the dimensionality of samples.A margin based fea- ture extraction algorithm is proposed for the problem.In view of the facts that for the high-di- mensional data,the probability of linear separability may grow in case of small samples and the low-dimensional projection is approximately normal,the proposed algorithm introduces a new definition of the margin,which involves not only the between-class scatter and within-class scat- ter proposed by LDA criterion,but also the differences of the class variances.Through maximal- izing the margin,we can obtain the optimal projection vector,and avoid the small sample size problem.Through theoretical analysis,the algorithm is further extended to the multi-class case. The experiment results show that the algorithm outperforms several improved versions of LDA in the case of small samples.At the same time,a satisfying performance is also achieved for larger samples.