Point-to-Point and Barrier Synchronization in Distributed SPMD Systems
Srđan Milaković · Rice Research Repository (Rice University) · 2019
Distributed memory programming models are very often the only way to scale up large scientific applications. To ensure correctness and optimal performance in distributed applications, it is necessary to use general, high-level, but efficient synchronization constructs. Implementing distributed applications using one-sided communication libraries is getting more popular, as opposed to the two-sided communication used in the MPI model. However, in most cases, those libraries only have support for high-level collective barrier synchronization and low-level point-to-point synchronization. Phaser synchronization construct is a very attractive synchronization mechanism because it unifies collective and point-to-point synchronization in a simple, easy to use high-level synchronization construct. In this thesis, we propose several novel algorithms for phaser synchronization on distributed-memory systems with one-sided communication. We also present several improvements to the distributed barrier algorithms in the OpenSHMEM reference implementation. We establish a very high confidence level in algorithms' correctness by using the SPIN model checker for our algorithms. We evaluated our phaser algorithm using several benchmark applications on large supercomputers, and we show that using phasers can reduce the synchronization time by up to 47% and improve total execution time by up to 26%. This thesis shows that high-level, efficient, and intuitive synchronization is possible on distributed systems with one-sided communication.