Research on Maximum Benefit Problem in a MapReduce Cluster
Xi Wang · Chinese Journal of Computers · 2015
MapReduce is one of the most popular parallel systems for big-data analysis.Many companies have built their MapReduce clusters to provide computing services to users.Users can submit their deadline-constraint MapReduce jobs to the cluster.If the jobs are finished before their deadlines,the company can get some benefits.For this application scenario,the maximum benefit problem in a MapReduce cluster is firstly presented in this paper.To solve this problem effectively,a sequence-based task scheduling strategy(SEQ strategy for short)is proposed,and we prove the advantages of SEQ strategy for the deadline-constraint job processing.Based on SEQ strategy,a novel Algorithm for Maximum Benefit,AMB,is proposed.AMB can efficiently determine the acceptable jobs and provide the effective execution strategy which can maximize the benefit.Besides,for the exceptions(e.g.node failure)in practical applications,a timeouthandling method is proposed,which can further improve the practicality of the algorithm.At last,the effectiveness of the proposed algorithm is verified through plenty of experiments.