ENTROPY-BASED PRINCIPLE AND GENERALIZED CONTINGENCY TABLES
Vincent Vigneron · 2006
Abstract. It is well known that the entropy-based concept of mutual information provides a measure of dependence between two discrete random variables. There are several ways to normalize this measure in order to obtain a coefficient similar e.g. to Pearson’s coefficient of contingency. This paper presents a measure of independence between categorical variables and is applied for clustering of multidimensional contingency tables. We propose and study a class of measures of directed discrepancy. Two factors make our divergence function attractive: first, the coefficient we obtain a framework in which a Bregman divergence can be used for the objective function; second, we allow speciafication of a larger class of constraints that preserves varous statistics. 1 Formulation and analysis Clustering is the problem of partitoning a finite set of points in a multidimensional space into classes (called clusters) so that points belonging to the same class are similar. An important step in designing a clustering technique is defining a way to measure the quality of partitioning in terms of the above objective. Given such a measure, an