A survey of numerical linear algebra methods utilizing mixed-precision arithmetic
Ahmad Abdelfattah, Hartwig Anzt, Erik G. Boman, Erin Carson, Terry Cojean, Jack J. Dongarra, Alyson Fox, Mark Gates, Nicholas John Higham, Xiaoye Sherry Li, Jennifer A. Loe, Piotr Łuszczek, Srikara Pranesh, Siva Rajamanickam, Tobias Ribizel, Barry Smith, Kasia Świrydowicz, Stephen James Thomas, Stanimire Z. Tomov, Yaohung M. Tsai · The International Journal of High Performance Computing Applications · 2021
The efficient utilization of mixed-precision numerical linear algebra algorithms can offer attractive acceleration to scientific computing applications. Especially with the hardware integration of low-precision special-function units designed for machine learning applications, the traditional numerical algorithms community urgently needs to reconsider the floating point formats used in the distinct operations to efficiently leverage the available compute power. In this work, we provide a comprehensive survey of mixed-precision numerical linear algebra routines, including the underlying concepts, theoretical background, and experimental results for both dense and sparse linear algebra problems.