Design and implementation of message-passing services for the Blue Gene/L supercomputer
George S. Almasi, Charles J Archer, José G. Castaños, John A. Gunnels, C. Christopher Erway, P. Heidelberger, Xavier Martorell, José Eduardo Moreira, K. W. Pinnow, Joe Ratterman, Burkhard Steinmacher-Burow, William Gropp, Brian R. Toonen · IBM Journal of Research and Development · 2005
The Blue Gene®/L (BG/L) supercomputer, with 65,536 dual-processor compute nodes, was designed from the ground up to support efficient execution of massively parallel message-passing programs. Part of this support is an optimized implementation of the Message Passing Interface (MPI), which leverages the hardware features of BG/L. MPI for BG/L is implemented on top of a more basic message-passing infrastructure called the message layer. This message layer can be used both to implement other higher-level libraries and directly by applications. MPI and the message layer are used in the two BG/L modes of operation: the coprocessor mode and the virtual node mode. Performance measurements show that our message-passing services deliver performance close to the hardware limits of the machine. They also show that dedicating one of the processors of a node to communication functions (coprocessor mode) greatly improves the message-passing bandwidth, whereas running two processes per compute node (virtual node mode) can have a positive impact on application performance.