Deadline constrained Cost Effective Workflow scheduler for Hadoop clusters in cloud datacenter
S R Rashmi, Anirban Basu · 2016
Cloud computing has attracted many enterprises owing to its pay as go model and highly scalable resources. Many enterprises exploit cloud infrastructure not only for storage but also for its analytics. Currently, one of the well accepted key for analytical platform is Hadoop on datacenter. Cloud vendors host Hadoop clusters on the datacenter to provide high performance analytical computing facilities to its customers. As many concurrent users operate on the clusters for their jobs, scheduling should be very effective to complete the job in time and at same time use the resources efficiently with effective cost management. Various schedulers such as FIFO, fair share, capacity schedulers etc‥, are available for the Hadoop platform. These schedulers view jobs as a separate scheduling entity. And not many schedulers, schedule the workflow on datacenter. In this paper, we propose a Cost Effective Workflow scheduler for Hadoop and implement it. The job execution deadline and the financial costs are used to measure its performance.