Traditional High-Performance Computing with Container Technology (THPC)
Shin Leong, Craig S. Pohl · Practice and Experience in Advanced Research Computing · 2023
High-performance Computing (HPC) has been around for decades but maintaining software for HPC with heterogeneous compute nodes remains a challenging task for system administrators. Many software package frameworks have been developed over the years to ease software installation tasks, for example, EasyBuild [1], Spack [2], and GNU Guix [3]. Advances in virtualization technology, such as hypervisor and container, have dramatically changed the computing world. Incorporating container technology into HPC has its own challenges, not to mention adding HPC software on top of container technology in an HPC cluster. This paper will describe Washington University in St. Louis (WashU) Traditional High-Performance Computing (THPC) which utilizes container technology, software package frameworks, and IBM Spectrum LSF 10.1 [14] as an HPC job scheduler. We will describe the WashU research HPC environment architecture, THPC system configurations, THPC software packages, and THPC use cases.