Algorithms and Data Structures for Matrix-Free Finite Element Operators with MPI-Parallel Sparse Multi-Vectors
Denis V. Davydov, Martin Kronbichler · ACM Transactions on Parallel Computing · 2020
Traditional solution approaches for problems in quantum mechanics scale asO(M3), whereMis the number of electrons. Various methods have been proposed to address this issue and obtain a linear scalingO(M). One promising formulation is the direct minimization of energy. Such methods take advantage of physical localization of the solution, allowing users to seek it in terms of non-orthogonal orbitals with local support. This work proposes a numerically efficient implementation of sparse parallel vectors within the open-source finite element library deal.II. The main algorithmic ingredient is the matrix-free evaluation of the Hamiltonian operator by cell-wise quadrature. Based on ana-priorichosen support for each vector, we develop algorithms and data structures to perform (i) matrix-free sparse matrix multivector products (SpMM), (ii) the projection of an operator onto a sparse sub-space (inner products), and (iii) post-multiplication of a sparse multivector with a square matrix. The node-level performance is analyzed using a roofline model. Our matrix-free implementation of finite element operators with sparse multivectors achieves a performance of 157 GFlop/s on an Intel Cascade Lake processor with 20 cores. Strong and weak scaling results are reported for a representative benchmark problem using quadratic and quartic finite element bases.