Adaptive Information Filtering Using Evolutionary Computation
Daniel R. Tauritz, Joost N. Kok, Ida G. Sprinkhuizen-Kuyper · 1997
this paper we describe a case study whose purpose was to investigate whether evolutionary computation can be useful in AIF systems. In this case study we took as input articles from a fixed number of different Internet newsgroups and gave them to the system. The goal of the system is to cluster these articles in groups: that is, each article should be assigned a label which corresponds to a newsgroup. We propose an AIF system for this task based on the novel combination of weighted trigram analysis, incremental clustering, and evolutionary computation. A trigram is a combination of three symbols. Trigram analysis [3] consists of determining the frequency distribution of all trigrams in a textual document. An enhancement to trigram analysis is the assignment of weights to all the trigrams indicating their relative importance for discriminating between documents on different topics. This `weighted trigram analysis' was used in the AIF system described here. An incremental clustering algorithm is applied to weighted trigram representations of the documents creating a classification of the documents. An incremental clustering algorithm is required because AIF is a dynamic process. In incremental clustering the number of clusters is not determined in advance, and can change over time [6]. Also the prototype vectors corresponding to the clusters can move over time. To find the right weights for the trigram analysis, we designed an evolutionary algorithm. The most complex step in this AIF system is finding the (near) optimal weights for the trigrams: even using only the 26 letters of the Latin alphabet and the space as delimiter symbol, that still leaves 19683 weights to optimize. This optimization problem is user dependent, so it cannot be performed a priori, and must also be...