ReShape: A Universal Data Processing Approach for Clustering Inhomogeneous and Weakly Separated Manifold Data
Zhongju Shang, Yaoguo Dang, Jun Zhu, Haowei Wang, Junjie Wang · IEEE Transactions on Industrial Informatics · 2024
Classical clustering methods such as k-means, spectral clustering, DBSCAN, and DPC often operate under specific assumptions about data characteristics. However, the increasing diversity and complex characteristic of data, coupled with limited prior knowledge, hinder the direct application of these clustering algorithms. Existing data processing approaches, such as ReScale and CDF-TS, are capable of homogenizing cluster density, but tailor to density-based algorithms that does not perform well on weakly separated or manifold clusters. To overcome these issues, we propose a novel data processing method (ReShape) facilitating the direct application of well-known clustering algorithms without modifications. The ReShape integrates three key components: a homogenization factor for uniforming varying cluster densities, a separation factor for enhancing cluster separations, and a joint distance considering both global and local consistency to measure distances in manifold datasets. The proposed method is universally applicable, and can match implicit assumptions of existing clustering algorithms. Extensive experiments and a case study for fault diagnosis involving k-means, spectral clustering, DBSCAN, and DPC demonstrates the efficacy of the ReShape approach.