Categorical Clustering of Market Basket Data by Mutual Information
Bartholomäus Ende, Rüdiger S. Brause · SSRN Electronic Journal · 2008
Traditional clustering approaches base inherently on metrical distance measures. Thus, in cases where this proposition does not prevail, like e.g. categorical data, their results might become misleading or even erroneous. One example is binary market basket data possessing a varying number of attribute entries within its representation. This contribution analyzes the special requirements of clustering binary market basket data for the example of user proling. The data base is category information which is provided in our case by cooking recipes. Further, it outlines potential problems that might arise from metrical distance functions and standard clustering procedures. In order to overcome these shortcomings we introduce a new clustering approach which is based on mutual information between the samples and outline its implications. Moreover, validation methods for categorical clusters are introduced and applied to the computed clusterings. They also answer how to determine a suitable number of clusters, how to evaluate their separability and stability as well as a way to identify their most significant characteristics.