MapReduce Parallel Model Based on Tree Structure
Bin Tang · 2015
MapReduce is a distributed computing model introduced by Google,which has been widely used in the field of massive data processing.A novel MapReduce parallel model was presented in this paper.The model is suitable for massive scientific data analysis,using unreliable desktop PC resources in the Internet or Intranet environment.Computing nodes are organized in the form of P2 P,and the P2P-MPI framework is utilized in the lower layer,while message passing interface model is utilized to achieve the MapReduce application layer.In the implementation of MapReduce application layer,the way of broadcast is used to distribute data chunks in the Map stage,and an inverse binary tree is constructed to realize effective intermediate results reduction in the Reduce stage.The proposed MapReduce mode was compared with existing popular MapReduce modes.The results show that the proposed tree structure-based MapReduce parallel model has a good performance in terms of fault-tolerance and it is simple and easy for application development.