Clustering massive small data for IOT
Xin Tao, Chunlei Ji · 2014
Data of IOT (Internet of things) have characteristics of heterogeneity, massive, timeliness and other features, which indicate that much of its data is in the form of small files. Cloud computing is used to deal with large data sets, but a large number of small data sets in the system will occupy most of the resources, resulting in a waste of system resources. In this paper, according to the characteristics of the mass of small data sets, and the deficiency of HDFS handle huge amounts of small data sets. This paper uses MapReduce to analysis the numerous small data sets and proposes a cluster strategy for massive small data based on the k-means clustering algorithm. The experimental results show that the proposed strategy can improve the data processing efficiency, and can improve the utilization of system resources. The research fruits will help us to design more practical merger strategy of massive small data to provide research reference.