Design and Implementation of Job Scheduling Algorithm for Multi-User MapReduce Clusters
Shuqiang Yang · Computer and Modernization · 2010
As more enterprises start to use data-intensive cluster computing systems such as Hadoop and Dryad for more applications,sharing MapReduce clusters among multiple users that reducing the cost of establishing an independent cluster and the demand of sharing common data sets resources for users is increasing.Based on fair scheduling algorithm,combining with slot allocation delay and priority technology,the paper proposes an improved algorithm.It can achieve better data locality,improve the performance of the system,such as throughput,response time.To meet the differentiated business services,it sets the appropriate for users to ensure special tasks.