ETP4HPC SRA White Paper - Programming Environment
Carpenter, Paul M., Gabriel Antoniu, Manuel Arenaz, Olivier Aumage, Jakub Beránek, Alfredo Buttari, Alexandru Costan, Sonja Happ, Venkatesh Kannan, Christian Pérez, Antonio J. Peña, Alberto Scionti, Xavier Vigouroux, VIVIANI, PAOLO · HAL (Le Centre pour la Communication Scientifique Directe) · 2024
This is a white paper released as part of the ETP4HPC’s Strategic Research Agenda 6. High-performance computing (HPC) applications achieve extreme levels of performance on large-scale systems by utilizing a wide range of tools, including compilers, runtime/middleware, APIs for memory access, debuggers and performance profilers, as well as high-level frameworks and domain-specific languages (DSLs). Modern HPC programming environments also increasingly incorporate AI software development tools and workflow management systems. The programming environment must ensure that existing codebases, many developed over decades in languages like Fortran, are compatible with diverse hardware architectures, such as CPUs, GPUs, AI accelerators, and more, as well as having to adapt to different and heterogeneous operational conditions in the computing continuum. Since the publication of ETP4HPC’s SRA5 in October 2022, the most significant advancement in software development has been the rise of generative AI, particularly the use of large language models such as ChatGPT and Microsoft Copilot. More broadly, the rapid advance in AI presents several challenges and opportunities across the HPC programming environment, in four key areas: (a) a growing share of HPC workloads now incorporates AI, (b) generative AI tools are increasingly being used to assist developers throughout the software development process, c) AI techniques are enhancing existing tools for code optimization, code generation and runtime optimizations, and (d) as the broader market prioritizes the demands of AI training and inference, there is a risk that hardware may start to neglect support for essential HPC features. As the number of cores and accelerator devices scale and memory and compute heterogeneity increases, the programming environment must improve its support for performance portability and programmer productivity. This goal, highlighted in the SRA since the first edition, remains an open topic of research. Management of heterogeneous memories is a particular concern. Task-based programming systems are the best way to keep the parallelism manageable, but they require a co-design effort between applications and programming models that takes account of the complexity of HPC applications. The programming environment should enable an incremental development process to discover parallelism, as well as optimizing the task grain and providing an agile and flexible mapping of parallelism to execution units. As HPC becomes more integrated into the Computing Continuum, programming models, environments, and performance portability are being reshaped to accommodate a more heterogeneous landscape. This integration underscores the growing importance of interoperability, composability, and standardization. Moreover, power efficiency has emerged as a critical challenge, necessitating innovative approaches to manage energy consumption effectively. As the post-exascale era emerges, the above trends are expected to become even more pronounced, leading to hard challenges for cross-cutting methods, algorithms, and software production.