An Information Foraging Approach to Determining the Number of Relevant Features
Brian Connolly, Benjamin Glass, John P Pestian · 2016
For many types of high-dimensional data, such as natural language corpora, the vast majority of extracted variables or features are essentially noise.Culling such features can not only reveal important patterns, but also improve the performance of supervised and unsupervised machine algorithms.Most research on feature selection has focused on the statistical measures used to rank features.Meanwhile, little work has been done developing techniques for identifying the optimal subset of features without repeatedly training models.However, developing such techniques is important, as they can significantly decrease computation time while providing a way to determine the features that characterize the classes within a data set, independent of how the data may be classified in the future.Here we introduce a novel method based on information foraging that works in conjunction with existing feature ranking methods to automatically determine a subset of important features.The method is demonstrated on simulated and linguistic data from psychiatric interviews.We show that the method is able to accurately determine the features that characterize the classes within both data sets.The method is fast, simple, and independent of any method of classifying the data, and can be extended to any highdimensional data set.