An improved data load balancing algorithm for hadoop
Niu Wen-liang · Journal of Henan Polytechnic University · 2013
This paper first introduces the principle of Hadoop and HDFS.Then the algorithm of Hadoop data load balancing is analyzed.This Hadoop algorithm balances the data according to the space usage of each node and does not handle the factors as processing power,bandwidth,files' access frequency.Thusly there is a big difference in response time for the similar files.This paper devises a novel load balancing model based on the factors of files' size,files' concurrent access time,files' access frequency,nodes' processing power,bandwidth and nodes' available storage space.Experimental results show that the devised model cannot only guarantee the storage space load balancing,but also make the similar files' response time more consistent.