RAJA Performance Suite: Performance Portability Analysis with Caliper and Thicket
Olga Pearce, Jason Burmark, Rich Hornung, Befikir Bogale, Ian Gordon Lumsden, Michael McKinsey, Dewi Yokelson, David Boehme, Stephanie Brink, Michela Taufer, Tom Scogland · 2024
Maintaining performant code in a world of fast-evolving computer architectures and programming models poses a significant challenge to scientists. Typically, benchmark codes are used to model some aspects of a large application code’s performance, and are easier to build and run. Such benchmarks can help assess the effects of code or algorithm changes, system updates, and new hardware. However, most performance benchmarks are not written using a wide range of GPU programming models. The RAJA Performance Suite provides a comprehensive set of computational kernels implemented in a variety of programming models. We integrated the performance measurement and analysis tools Caliper and Thicket into the RAJA Performance Suite to facilitate performance comparison across kernel implementations and architectures. This paper describes the RAJA Performance Suite, performance metrics that can be collected, and experimental analysis with case studies. The RAJA Performance Suite and integration of Caliper and Thicket enabled us to effectively measure the variety of implementations on the variety of hardware, and automatically characterize subsets of kernels which exhibit similar bottlenecks—and therefore perform similarly on new architectures which provide a different balance between resources such as FLOPS and memory bandwidth. We definitively demonstrate that the most memory bound kernels show the most performance gains on architectures with high-bandwidth memory (HBM), and that the kernels that have other bottleneck may, to a lesser extent, benefit from the higher-FLOPS GPUs.