Using Unsupervised Learning to Guide Resampling in Imbalanced Data Sets
Adam Nickerson, Nathalie Japkowicz, Evangelos Milios · 2001
The class imbalance problem causes a classier to over- t the data belonging to the class with the greatest number of training examples. The purpose of this paper is to argue that methods that equalize class membership are not as e ective as possible when applied blindly and that improvements can be obtained by adjusting for the within-class imbalance. A guided resampling technique is proposed and tested within a simpler letter recognition domain and a more di cult text classi cation domain. A fast unsupervised clustering technique, Principal Direction Divisive Partitioning (PDDP), is used to determine the internal characteristics of each class. The performance improvement in categories that su er from a large between-class imbalance (few positive examples) are shown to be improved when using the guided resampling method. 1