Performance Enhancement of MapReduce Framework in Big Data Application Using Load Balancing with Cache
International Journal of Science and Research (IJSR) · 2015
Hadoop is open source software that is used to store big data, it supports data demanding applications and performs analysis, using a random placement method for parallel processing to give effortlessness and load balance.To achieve maximum parallelism per group to load balance a new Data-gRouping-AWare (DRAW) data placement is used.Problem in big data is when any query executes repeatedly it repeats whole process of execution to obtain result.In MapReduce framework and generates a large amount of intermediate data.Such huge amount of information is thrown away after the tasks finish, because MapReduce is not able to use this data.Dache, a data-aware cache framework for big-data applications gives the produced intermediate results to the cache manager.Task inquiries the cache manager before performing the actual computing work.