Fringe SVM Settings and Aggressive Feature Reduction
Adam P. Kowalczyk, Bhavani Raskutti · 2003
Statistical techniques for aggressive feature reduction are studied on data obtained in agene knock-out experiment. The essential part of the process is automatic assessment of the quality of various feature selection methods. This is done by comparison of the performance of discriminating models built on candidate subsets of features. Experiments show that typical settings of popular 2-class discriminators, support vector machines (SVM), cannot be used as they produce models of very poor quality. The proposed way around is to use “fringe classifiers” such as SVMs trained on positive class data only or class centroids. Additionally, we also use models generated by such algorithms directly for identification of most discriminating features. We recommend that such simple machine learning techniques should be included into arepertoire of discrimina tors used on such occasions. We show that such relatively superior performance of fringe SVMs can also be observed on regular textmining data bases, such as Reuters newswire benchmark, if only the less frequent features (words) are used.