Subset barrier synchronization on a private-memory parallel system
Anja Feldmann, Thomas R. Gross, David R. O’Hallaron, Thomas M. Stricker · 1992
A global barrier synchronizes all processors in a parallel system.This paper investigates algorithms that allow disjoint subsets of processors to synchronize independently and in parallel.The user model of a subset barrier is straight forward; a processor that participates in a subset barrier needs to know only the name of the barrier and the number of participating processors.This paper identifies two general communication models for private-memory parallel systems: the bounded buffer broadcast model and the anonymous destination messagepassing model and presents algorithms fior barrier synchronization in the terms of these models.The models are detailed enough to allow meaningful cost estimates for their primitives, yet independent of a specific architecture and ~canbe supported efficiently by a modem private-memory parallel system.The anonymous destination message passing model is the most attractive.The time complexity to synchronize over a uni-directional ring of N processors is O(log N) for common cases, and 0( m) in the worst case.The algorithms have been implemented on iWarp, a private-memory parallel system and are now in daily use.The paper concludes with timing measurements obtained on a 64-node system. 1 Introduction Barrier synchronization is a useful technique for organizing 'the execution of a parallel program into a sequence of loosely coordinated phases.For example, a phase of a data-parallel program running on a private memory system might consist of a communication step,