Kernel-Assisted MPI Collective Communication among Many-core Clusters
Teng Ma · 2012
Architectural hierarchies and hardware complexity brought by multicore or many-core Clusters, greatly challenge MPI applications' performance in two ways: performance efficiency and cross-platform portability. The cross-platform portability assumption, 'write once and efficiently run everywhere,' is not guaranteed by current MPI libraries, mainly due to implementation details. To partially address the performance issue exposed by hardware complexity and memory hierarchies, we propose a kernel assisted MPI collective communication approach, directly based on the kernel assisted one-sided single-copy module: KNEM. First, we introduce the general operating principles of KNEM memory copy, and then we present the design and implementation of KNEM collective, an intra-node collective component of Open MPI for shared memory nodes. Additionally, we describe how to integrate the kernel-assisted approach into collective communications on heterogeneous multicore clusters with intra- and inter-node communication. We evaluate and experimentally demonstrate the performance advantages of our kernel-assisted MPI collective over state-of-art MPI libraries (Open MPI and MVAPICH2).