Stochastic Model Predictive Control with Direct Feedforward Compensation: Harvesting Idle Resources in High-Performance Computing
Kouds Halitim, Bogdan Robu, Sophie Cerf, Raphaël Bleuse, Éric Rutten · 2025
In high-performance computing (HPC), many research has been conducted on how to efficiently utilize the idle time of HPC resources (periods when no jobs are submitted to the platform). One promising approach is to exploit this idle time by injecting small, flexible, independent, and interruptible jobs that have no strict time constraints. However, managing the injection of these jobs is challenging due to the stochastic nature of job parameters, such as execution times and resource consumption. Additionally, process noise—resulting from system complexity and the arrival and execution of varying external workloads—can interrupt or terminate these filler jobs. In this paper, we propose and evaluate, using real data, a Stochastic Model Predictive Control (SMPC) approach that addresses system uncertainty and incorporates a feed-forward compensation mechanism for disturbance rejection. The proposed algorithm shows promising results: it ensures a platform usage rate of 98%, significantly improving overall resource efficiency and reducing the number of early terminated jobs compared to previous work.