Load-aware replica placement in multiuser Hadoop environment using MST
Amrita Patole, S. D. Madhu Kumar, Priya Chandran, T.P. Shabeera · 2015
In recent years, Hadoop framework is popularly known for providing cost-effective solutions to process large-scale data intensive applications in a distributed manner. Storage imbalance during replica placement in Hadoop is harmful. Replica placement in HDFS plays a major role in data availability and balanced utilization of clusters. In this paper we propose a solution for load-aware replica placement in Hadoop such that a cluster is divided into small size partitions where at-least one partition will be nearer to its users. Partitions are created using minimum spanning tree and results are compared with the default replica placement policy of Hadoop. Experimental results of the proposed solution confirm that, load-aware replica placement gives uniform rack level utilization and reduces read access time over the default replica placement policy of Hadoop in a multiuser environment.