Rule-based noise detection for software measurement data
Taghi M. Khoshgoftaar · 2005
The quality of training data is an important issue for classification problems, such as classifying program modules into the fault-prone and not fault-prone groups. The removal of noisy instances will improve data quality, and consequently, performance of the classification model. We present an attractive rule-based noise detection approach, which detects noisy instances based on Boolean rules generated from the measurement data. The proposed approach is evaluated by injecting artificial noise into a clean or noise-free software measurement dataset. The clean dataset is extracted from software measurement data of a NASA software project developed for realtime predictions. The simulated noise is injected into the attributes of the dataset at different noise levels. The number of attributes subjected to noise is also varied for the given dataset. We compare our approach to a classification filter, which considers and eliminates misclassified instances as noisy data. It is shown that for the different noise levels, the proposed approach has better efficiency in detecting noisy instances than the C4.5-based classification filter. In addition, the noise detection performance of our approach increases very rapidly with an increase in the number of attributes corrupted.