Data preparation techniques for improving rare class prediction
Nittaya Kerdprasop, Kittisak Kerdprasop · International Conference on Mathematical methods, Computational techniques and Intelligent systems · 2011
Rare class prediction is the data mining task aiming at building a model that can correctly identify objects or events rarely occurring in the data set. In many real life applications such as identification of intruders accessing a network system, detecting fraudulent credit card transactions, it is rare events that are of great interest. Unfortunately, traditional mining algorithms fail to predict rare events because the model are inherently built in favor of the majority class to draw common characteristics among data instances. Rare class mining is thus a challenging problem in some specific domains. We study the rare class mining problem in the context of semiconductor manufacturing process control in which fault products are rarely occurred, but once occurring they require timely detection to prevent the decrease in product yield. In this paper, we propose to use an over-sampling technique to alleviate the outnumber situation of majority class. Such sampling technique is however prone to introducing the over-fitting problem. We thus propose the remedy by applying the cluster based technique to selectively extract data instances showing discrimination characteristics. The built models from various mining algorithms have been tested with a separate data set and the results show significant improvement on the predicting accuracy.