A scalable method for instance selection for class-imbalance datasets
Aida de Haro-García, Nicolás García‐Pedrajas · 2011
Instance selection is becoming more and more relevant due to the huge amount of data that is constantly being produced. Research areas such as bioinformatics, text mining and intrusion detection, are generating huge amounts of information that must be dealt with. Instance selection is a powerful tool to reduce that information to manageable datasets. Most of the datasets in these areas shares a common property, they are heavily class-imbalanced. The class of interest, or positive or minority class, is outnumbered many times by the negative, or majority, class. Thus, any instance selection algorithm addressing these problems must take into account two important features of such problems. Firstly, the large size of the datasets that makes scalability issues very relevant. Secondly, the class-imbalanced distribution of the instances. In this paper, we propose a new methodology for instance selection that it is specifically designed for large class-imbalanced datasets. We use a divide-and-conquer approach to deal with the scalability of the algorithms, and a combination of different rounds of instance selection to improve the results in terms of class-imbalance error measures. The validity of the proposed framework is assured using 45 datasets. Our proposal improves the results of standard methods in accuracy and storage reduction, and at the same time is able to reduce the time needed by the algorithms with a time complexity O(n log(n)).