Communication-Efficient Parallel Dense LU Using a3-Dimnsional Approach.
Dror Irony, Sivan Toledo · PPSC · 2001
We present new communication-efficient parallel dense linear solvers: An LU factorization algorithm and a triangular linear solver. The new algorithms perform asymptotically a factor of P 1/6 less communication than existing algorithms, where P is the number of processors . The new algorithms employ a 3-dimensional (3D) approach, which has been previously applied only to matrix multiplication. We have implemented and tested the new algorithms. Our LU factorization algorithm is competitive with ScaLAPACK and scales better with the number of processors. The new algorithms employ a 3D approach that reduces communication using replication. The algorithms perform less communication but use more temporary storage than existing algorithms, which all use a 2-dimensional (2D) approach. Until now, the 3D approach has only been used for parallel matrix multiplication in algorithms that were proposed by Berntsen [3], by Aggarwal, Chandra, and Snir [2], by Gupta and Kumar [5], by Johnsson [7], and by Agarwal, Balle, Gustavson, Joshi, and Palkar [1]. 3D algorithms work by distributing the 3D iteration space of the computation among processors. Matrix-matrix computations that can be implemented using three nested loops have a natural representation on a 3D grid in which every grid