Validating a data mining tool based upon a genetic classifier for segmenting tabular data
Ken Lunn · 1996
A genetic classifier has been developed which can partition tabular data based on the aggregate value of an attribute. The original motivation was to search for interesting conditions on market data that correlated with high profit retail outlets, thus identifying potentially interesting market segments as part of a retail market analysis. The genetic algorithm has also been applied to the identification of buy-sell signals for stocks and bonds, and the basic algorithm has a potentially wide range of application in areas where there are large quantities of data which can be used to determine relationships between sets of attributes. The genetic algorithm determines sets of conditions that select records in the data to give subsets of the records known as segments. Segments are valued according to the mean value of a particular attribute of the records in the segment, and subject to some minimum segment size. The fitness function is simple, but computationally expensive, and considerable effort has gone into optimising its performance. The genetic algorithm was first implemented in Smalltalk, and produced very good results, but at too slow a rate; the algorithm is now implemented in C++ with a significant speed improvement. It is now proposed to validate the method as a general purpose data mining tool and provide a Windows user interface. The end result should be an efficient and user friendly PC based data mining tool which will have been proven against a range of `real world' data sets. (8 pages)