Combining Feature Selection and Ensemble Learning for Software Quality Estimation
Kehan Gao, Taghi M. Khoshgoftaar, Randall Wald · The Florida AI Research Society · 2014
High dimensionality is a major problem that affects the quality of training datasets and therefore classification models. Feature selection is frequently used to deal with this problem. The goal of feature selection is to choose the most relevant and important attributes from the raw dataset. Another major challenge to building effective classification models from binary datasets is class imbalance, where the minority class has far fewer instances than the majority class. Data sampling (altering the dataset to change its balance level) and boosting (building multiple models, with each model tuned to work better on instances misclassified by previous models) are common techniques for resolving this problem. In particular, ensemble boosting, which integrates sampling with AdaBoost, has been shown to improve classification performance, especially for imbalanced training datasets. In this paper, we investigate approaches for combining feature selection with this ensemble learning (boosting) process. Six feature selection techniques and two forms of the ensemble learning method are examined. We focus on two different scenarios: feature selection performed prior to the ensemble learning process and feature selection performed inside the ensemble learning process. The experimental results demonstrate that performing feature selection inside of ensemble boosting generally performs better than using feature selection prior to ensemble boosting.