Towards Predictive Modeling of Message-Passing Communication
Verdi March, Vijayaraghavan Murali, Yong Meng Teo, Simon Chong Wee See, James T. Himer · 2009
Communication has been shown to be a performance bottleneck and a limiting factor of many large parallel applications. As such, predicting the application scalability necessitates a communication performance model. This paper investigates the LogGP communication performance model for predicting message-passing communications when the system configuration (i.e., number of nodes) is varied. The cost functions for the message-passing operations are based on MVAPICH2 1.0, and the experiments are conducted on the Ranger system using up to 256 nodes connected with an InfiniBand net-work. For point-to-point communications, we observe that the LogGP model accurately predicts the communication performance. However, the results for three collective operations, i.e.,MPI_Barrier, MPI_Alltoall, andMPI_Bcast, are varying. ForMPI_Bcast, the LogGP model is able to predict its scalability up to 256 nodes, and the prediction error is at most a factor of two on 256 nodes. For the remaining collectives, the scalability— bar that ofMPI_Alltoallon small messages (m= 2 bytes) — is predicted by LogGP, but the prediction error for 256 nodes is 3.5-12 times of the measured performance.