Versatile Software-Defined Cluster for HPC Using Cloud Abstractions
Maxime Martinasso, Mark Klein, Benjamin Cumming, Miguel Gila, Felipe A. Cruz, Alberto Madonna, Manuel Sopena Ballesteros, Sadaf R. Alam, T. C. Schulthess · Computing in Science & Engineering · 2024
This paper introduces the versatile software-defined cluster, a novel framework that integrates HPC and Cloud technologies offering a service-oriented approach for computing resources instead of a hardware focused one, maintaining infrastructure independence and avoiding vendor lock-in. It addresses the challenges of rigidity and lack of customizability in conventional HPC systems, facilitating a more efficient use of shared infrastructures. The core concept revolves around a three-tiered structure—Infrastructure, Service Management, and Software-as-a-Service—ensuring immutable and consistent service deployment for the diverse scientific communities. This integration enhances scientific workflows’ adaptability and efficiency, enabling Science-as-a-Service with cost-effective solutions for shared digital research infrastructure with diverse usage patterns, such as batch, interactive, urgent computing or machine learning workflows.