Research on optimization of K-means Algorithm Based on Spark
Menghua Han · 2023
This paper studies the principle and implementation of the commonly used clustering algorithm K-means and K-means ++, analyzes the advantages and disadvantages of the algorithm, and improves the K-means ++ algorithm based on the idea of dynamic adjustment of the clustering center for the problem that the initialization center may be uneven. The improved algorithm improves the clustering effect by removing the worst cluster center and splitting the cluster center to be split. And this paper implements the parallelization of the improved algorithm based on Spark. Experiments on clustering effect, running time and acceleration ratio show that compared with the traditional K-means ++ algorithm, the improved K-means ++ algorithm not only improves the quality of clustering and reduces the computing time, but also shows good parallel performance in the multi-node cluster environment. The experimental results show that the proposed algorithm can effectively improve the efficiency of algorithm execution and parallel computing ability.