The Optimization of Memory Access Congestion for MapReduce Applications on Manycore Systems
Liang Li, Endong Wang, Xiaoshe Dong, Zhengdong Zhu · The Computer Journal · 2014
The prevalence of heterogeneous manycore processors has been shown as a promising alternative to web-based data parallel MapReduce applications. The differences of manycore from multicore raise new challenges to designing and implementing efficient and scalable MapReduce applications, such as memory access congestion. This paper argues that it is more efficient using multiple small groups of cores than using a single group with all cores to address these challenges. We propose a group-based MapReduce implementation, called Grouped-MapReduce (GMR). It uses grouping to align tasks so that the tasks in each group are congestion-free. Further, it uses multiplexing to control the implementation of groups so that multiple groups can efficiently share the limited memory bandwidth without causing serious memory access congestion. We have implemented a prototype of GMR based on Phoenix++, an already highly optimized MapReduce runtime. Experiments on six benchmarks show that GMR implements and scales well on manycore systems and obtains an impressive improvement over Phoenix++ from 1.04x to 1.77x without artificially tuning the existing application code.