Exploiting ILP, DLP, TLP, and MPI to accelerate matrix multiplication on Xeon processors
Mostafa I. Soliman, Fatma S. Ahmed · 2014
Matrix multiplication is one of the most important kernels used in the dense linear algebra codes. It is a computational intensive kernel that demands exploiting all available forms of parallelism to improve its performance. In this paper, ILP, DLP, TLP, and MPI are exploited to accelerate the execution of matrix multiplication on a cluster of computers with Intel Xeon processors. In addition, the Strassen's algorithm, which reduces the arithmetic operations of matrix multiplication from O(n3) to O(n2.807), is implemented using multi-threading, SIMD, blocking, and MPI techniques. Our results show that the average speedups of the traditional matrix multiplication algorithm on large matrices (from 6000×6000 to 9000×9000) are 14.9, 24.1, 24, 22.8 and 20.7 on 2, 4, 6, 8, and 10 computers, respectively. However, on the Strassen's algorithm, the average speedups are 3.9, 6.8, 6.8, and 10.3 on 2, 4, 6, and 7 computers, respectively, by applying the same parallel processing techniques.