A Modified KS-test for Feature Selection
IOSR Journal of Computer Engineering · 2013
A central problem in machine learning is identifying a representative set of features from which to construct a classification model for a particular task.Feature selection, as a preprocessing step to machine learning, is effective in reducing dimensionality, removing irrelevant data, increasing learning accuracy, and improving result comprehensibility.The central hypothesis is that good feature sets contain features that are highly correlated with the class, yet uncorrelated with each other.In this paper a fast redundancy removal filter is proposed based on modified Kolmogorov-Smirnov statistic, utilizing class label information while comparing feature pairs.Results obtained from this algorithm are compared with other two algorithms capable of removing irrelevancy and redundancy, such as Correlation Feature Selection algorithm (CFS) and simple Kolmogorov Smirnov-Correlation Based Filter (KS-CBF).The efficiency and effectiveness of various methods is tested with two of the standard classifiers such as the Decision-Tree classifier and the K-NN classifier.In most cases, classification accuracy using the reduced feature set produced using the proposed approach equaled or bettered accuracy obtained using the complete feature set and other two algorithms.