TRANSFER-ATTENTIVE DIVISION AND AGGREGATION INCHART DECREASE FOR BIG DATA APPLICATIONS
K. Durgarao, K . Vijaya Bhaskar · IJITR International Journal of Innovative Technology and Research - IJITR International Journal of Innovative Technology and Research · 2017
In this paper, we study to reduce network traffic cost for virtually any Map Reduce job by developing a manuscript intermediate data partition plan. In addition, we with each other consider the aggregator positioning problem, where each aggregator helps to reduce merged traffic from multiple map tasks. However, some attempts are actually made to enhance the performance of Map Reduce jobs, they ignore the network traffic created inside the shuffle phase, which plays a crucial role in performance enhancement. The Map Reduce programming model simplifies large-scale computer on commodity cluster by exploiting parallel map tasks minimizing tasks. Finally, extensive simulation results show our plans can significantly reduce network traffic cost under both offline an internet-based-based cases. Typically, a hash function enables you to partition intermediate data among reduce tasks, which, however, is not traffic-efficient because network topology and understanding size connected with each and every single key aren't considered. A decomposition-based distributed formula is recommended to deal with big-scale optimization problem for giant data application with an online formula may also be designed to adjust data partition and aggregation inside the dynamic manner.