Balancing reducer skew in MapReduce workloads using progressive sampling
Smriti R. Ramakrishnan, Garret Swart, Aleksey Urmanov · 2012
The elapsed time of a parallel job depends on the completion time of its longest running constituent. We present a static load balancing algorithm that distributes work evenly across the reducers in a MapReduce job resulting in significant elapsed time reductions.