Parallel iterative solvers for unstructured grids using a directive/MPI hybrid programming model for the GeoFEM platform on SMP cluster architectures
Kengo Nakajima, Hiroshi Okuda · Concurrency and Computation Practice and Experience · 2002
Abstract In this paper, an efficient parallel iterative method for unstructured grids developed by the authors for shared memory symmetric multiprocessor (SMP) cluster architectures on the GeoFEM platform is presented. The method is based on a three‐level hybrid parallel programming model, including message passing for inter‐SMP node communication, loop directives for intra‐SMP node parallelization and vectorization for each processing element (PE). Simple 3D elastic linear problems with more than $10^8$ degrees of freedom have been solved by $3\times3$ block ICCG(0) with additive Schwarz domain decomposition and PDJDS/CM‐RCM reordering on 16 SMP nodes of a Hitachi SR8000 parallel computer, achieving a performance of 20 Gflops. The PDJDS/CM‐RCM reordering method provides excellent vector and parallel performance in SMP nodes, and is essential for parallelization of forward/backward substitution in IC/ILU factorization with global data dependency. The method developed was also tested on an NEC SX‐4 and attained 969 Mflops (48.5% of peak performance) using a single processor. The additive Schwarz domain decomposition method provides robustness for the GeoFEM parallel iterative solvers with localized preconditioning. Copyright © 2002 John Wiley & Sons, Ltd.