Container orchestration on HPC systems through Kubernetes
Naweiluo Zhou, Yiannis Georgiou, Marcin Pospieszny, Zhong Li, Huan Zhou, Christoph Niethammer, Branislav Pejak, Oskar Marko, Dennis Hoppe · Journal of Cloud Computing Advances Systems and Applications · 2021
Abstract Containerisation demonstrates its efficiency in application deployment in Cloud Computing. Containers can encapsulate complex programs with their dependencies in isolated environments making applications more portable, hence are being adopted in High Performance Computing (HPC) clusters.Singularity, initially designed for HPC systems, has become theirde factostandard container runtime. Nevertheless, conventional HPC workload managers lack micro-service support and deeply-integrated container management, as opposed to container orchestrators. We introduce a Torque-Operator which serves as a bridge between HPC workload manager (TORQUE) and container orchestrator (Kubernetes). We propose a hybrid architecture that integrates HPC and Cloud clusters seamlessly with little interference to HPC systems where container orchestration is performed on two levels.