A new cluster validity index for overlapping datasets
Erzhou Zhu, Xue Wang, Feng Liu · Journal of Physics Conference Series · 2019
The choice of the optimal clustering number ( K opt ) is very important for forming the clustering results. As clustering is a kind of unsupervised learning method, it is still difficult to determine the K opt . The clustering validity index (CVI) is an effective method for determining the K opt and evaluating the clustering results generated by clustering algorithms. However, many of the existing CVIs cannot efficiently with overlapping datasets. In this paper, we propose WCH, a new cluster validity index for overlapping datasets. The WCH index is constructed based on features of inner-cluster tightness, inter-cluster separation and inter-cluster overlapping of datasets. The WCH index can effectively process datasets with high overlapping and can accurately find the corresponding K opt . The comparison experiments are carried between our new WCH index and four existing CVIs on five simulation datasets. The experimental results have shown that the new WCH index is stable in forming the K opt for many kinds of overlapping datasets.