K Means of Cloud Computing: MapReduce, DVM, and Windows Azure
Lin Gu, Zhonghua Sheng, Xiang Gao, Chuan Zhang, Yaohui Jin · 2012
Abstract—Cloud-based systems and the datacenter computing environment present a series of challenges to system designers for supporting massively concurrent computation on clusters with commodity hardware. The platform software should abstract the unreliable but highly provisioned hardware to provide a high-performance platform for a diversity of concurrent programs pro-cessing potentially very large data sets. Toward this goal, a num-ber of solutions are designed or proposed. Among these products and systems, we elect three technologies, MapReduce/Hadoop, DVM, and Windows Azure, as representatives of three different approaches to constructing the infrastructure and instructing the programming in the cloud. We empirically study these technologies using a well-known and widely used application, k-means, and analyze their performance data in relation with the abstraction layers they establish. The implementations of k-means on the three platforms are presented with sufficient details to show the design patterns with these technologies. We analyze the evaluation results in the context of the design goals and constraints of the technologies, and show that the instruction-level abstraction can provide flexible programming capability as well as high performance. Keywords—Cloud computing; k-means; parallel programming; MapReduce; DISA; big data processing I.