Evaluation of Evolutionary Data Mining Techniques
Thomas V. Fernandez · 2004
To successfully develop new evolutionary data mining techniques, we need a methodology for evaluating their benefits. Evolutionary techniques are stochastic so experiments must be designed to report the results with a high level of statistical confidence. Clearly the evaluation of evolutionary data mining techniques must be based on the performance of the resulting classifier systems using observations from outside of the training data set. These experimental results should not be specific to a given database. The experiments should be repeatable using a variety of databases representing problems of different difficulty. During the experiments each technique being compared must be allocated the same amount of computational resources even if the resources are provided by multiple and dissimilar computers. In our development and testing of new evolutionary data mining techniques we have implemented a Genetic Programming system called NUGP designed for the statistical comparison of Genetic Programming techniques. To ensure the fairness of our experiments, we propose a new metric, the Number of S-expression Nodes Evaluated, to measure computational effort in Genetic Programming systems. We also implement a program called MAKDAT for generating sets of synthetic data with real world properties and different levels of difficulty.