Research on Optimization of Sorting Algorithm Based on MapReduce
Jiang Yon · Jisuanji kexue yu tansuo · 2015
MapReduce has become the standard parallel computing model on big data analysis. To balance highly the loading nodes in MapReduce system and minimize space usage, CPU, I/O operation time and network overhead, based on a good parallel algorithm, this paper proposes an optimization algorithm of MapReduce design specification, optimizing indexes at the same time. This paper also gives theoretical analysis for the most important sorting algorithm in data processing field, presents the optimal algorithm of multiple index constraints, and proves that the optimal algorithm meets the standard of MapReduce optimization algorithm. The experiments verify that this optimal sorting algorithm is better than the traditional sorting algorithm in terms of effectiveness and efficiency.