Communication and Optimization Aspects of Parallel Programming Models on Hybrid
Rolf Rabenseifner, Gerhard Wellein, Regionales Rechenzentrum Erlangen · 2002
Most HPC systems are clusters of shared memory nodes. Parallel programming must combine the distributed memory paralleliza- tion on the node inter-connect with the shared memory parallelization inside of each node. The hybrid MPI+OpenMP programming model is compared with pure MPI, compiler based parallelization, and other parallel programming models on hybrid architectures. The paper fo- cuses on bandwidth and latency aspects, but also whether programming paradigms can separate the optimization of communication and compu- tation. Benchmark results are presented for hybrid and pure MPI com- munication.