Real-Valued Jaccard and Coincidence Based Hierarchical Clustering
Luciano da Fontoura Costa · HAL (Le Centre pour la Communication Scientifique Directe) · 2021
Hierarchical clustering represents one of the most frequently adopted methodologies for identifying clusters in data in non-supervised classification tasks. Amongst the advantages of this family of approaches, we have that the possible solutions are obtained in a multiscale manner involving a respective dendrogram of the data. In addition to providing a more complete description of the interrelationships between the data elements, the number of clusters does not need to be specified as in other clustering methods such as k-means, as it can be inferred from the obtained dendrograms. There are several possible hierarchical clustering methods, depending on the adopted merging criterion, which can be the smallest distance between sets (single linkage), or the minimization of dispersion (Ward's). The Jaccard index has also be considered for binary data. In this work, we propose a new family of hierarchical clustering methods, based on recent developments in which the Jaccard index is generalized to real values as well as on the coincidence index, which corresponds to the product between this generalized index and the interiority (or homogeneity) index. The former of these indices is more discriminative of anti-correlations, and the latter also provides a more strict comparison of the involved clusters. Therefore, it is expected that the coincidence index-based hierarchical cluster be less likely to yield false positive clusters than other hierarchical approaches. In addition, it becomes possible to start with the elements to be clustered represented by generic densities or even general scalar fields.