Two-stage hierarchical clustering based on LSTM autoencoder
Zhihe Wang, Yangyang Tang, Hui Du, Xiaoli Wang, Zhiyuan Hu, Qiaofeng Zhai · 2022
Hierarchical clustering algorithm has low accuracy when processing high-dimensional data sets. In order to solve the problem, this paper presents a two-stage hierarchical clustering algorithm based on a Long Short Term Memory (LSTM) autoencoder. Firstly, we use LSTM autoencoder to learn the potential low-dimensional feature representation of the data. Secondly, we use the proposed two-stage hierarchical clustering algorithm to cluster the low-dimensional features. In the first stage, this algorithm divides the data points that are closest to each other into one cluster, for the data points have been clustered, their corresponding reverse nearest neighbors are also divided into the same cluster. In the second stage, the average density is defined as a measure, in each iteration, the cluster with the highest average density is merged with its nearest cluster until the threshold of the number of clusters is reached. Experiments on the UCI dataset show a significant improvement in the accuracy of the proposed algorithm when compared to the PERCH, BIRCH, CURE, SRC and RSRC algorithms.