Examining Fuzzy Cluster Validity Through the Lens of Ecological Diversity
Mohd Aquib, Dhan Jeet Singh, Teena Sharma, Nishchal Kumar Verma, M. Jaleel Akhtar · 2025
Cluster Validity Indices (CVIs) are essential for evaluating the performance of clustering algorithms, particularly in Fuzzy C-Means (FCM), where cluster quality depends on compactness and separation. However, conventional CVIs often fail to effectively handle the fuzziness, complex structures, or overlapping nature of clusters, resulting in less reliable evaluations. In this paper, we propose a novel CVI for FCM clustering by drawing inspiration from ecological diversity metrics to address the above limitations. Ecological diversity, a cornerstone concept in biodiversity studies, encapsulates the balance between species richness and their distribution across ecosystems. Analogously, we examine clustering quality as a balance between compactness (intra-cluster cohesiveness) and separateness (inter-cluster distinctiveness). The proposed CVI leverages Rényi entropy to quantify the fuzziness in cluster memberships, ensuring a nuanced assessment of cluster compactness. Additionally, we incorporate a Brillouin-inspired separateness measure that accounts for the spatial distance between cluster centroids and the overlap in memberships across clusters. This formulation intuitively parallels the ecological diversity framework, where higher diversity reflects balanced species distribution, just as high-quality clusters reflect well-separated and compact groupings. Experimental evaluation on benchmark datasets demonstrate that our CVI effectively captures the trade-off between compactness and separateness, serving as a promising metric to quantify clustering performance.