Distributed Clustering Algorithm for Network Data Based on MapReduce
Chen Dong-min · Jisuanji gongcheng · 2013
Due to the high time and space complexity and physical machines out of memory,traditional clustering algorithms usually can not effectively analyze and deal with large data network.To solve this problem,this paper proposes a distributed clustering algorithm for network data based on MapReduce model.It adopts the theory of MRC theory to design limited round number of MapReduce to control the time in shuffle stage,and utilizes the Map inner merging technology to control network flow.It proposes an idea that if merge the intermediate results,only merge clusters and do not consider the internal nodes,which can control memory overhead.It utilizes the data sets generated by simulation to do experiment.Experimental results show that when the data size and cluster scale increases,the CAMR algorithm has good speedup ratio and scalability.