Improving resource utilization in a heterogeneous cloud environment
Hsin-Yu Shih, Jenq‐Shiou Leu · 2012
Cloud computing features a flexible computing infrastructure for large-scale data processing. MapReduce is a typical model providing an logical framework for cloud computing and Hadoop, an open-source implementation of MapReduce, is a common platform to realize such kind of parallel computing model. Normally, a cloud computing service comprises many heterogeneous commodity machines. The original resource arrangement policy in Hadoop only focuses on the logical resources, such as free slot number, without considering the physical workload of comprehensive computing resources, such as the CPU utilization, network bandwidth, memory usage on each working node. This paper aims at dispatching the computation load to all processing nodes in the cloud computing environment by considering the physical workload on each node so as to prevent bias in arranging computation resources and hence improve the overall computing performance in a heterogeneous cloud environment.