A novel technique using slot configuration on Hadoop clusters
K. Hemalatha, G. Murali · 2016
The Hadoop tool is the real platform for optimal analysis for huge amount of data. How to reduce the completion length of a group of MapReduce jobs is one of the main things in Hadoop. The map and reduce are available in Hadoop tool implemented for processing and supply distributed large Terabyte data on capacity clusters. The present open source Hadoop allows only fixed slot configuration, like fastened of map slots and reduce slots entire the cluster lifetime. Such static configuration may lead to more completion time as well reduce the cpu utilizations. Its important responsibility is to shrink the completion time of huge sets of MapReduce jobs. It may be achieved by means of following slot ratio configuration between map and slash duties, via updating the workload know-how of lately completed duties. Many scheduling methodologies are discussed that purpose to make stronger completion time intention. Propose new schemes which use slot ratio between map and reduce tasks as a tunable knob for minimizing the completion length (i.e., makespan) of a given set. By leveraging and making a detailed Analysis on the workload of data collected in the form of information for a set of completed jobs, various schemes to be designed dynamically for allocation of various resources (or slots) to designing a mapping concept and reduce tasks allotment burden.