Scheduling and Allocation of Disaggregated Memory Resources in HPC Systems
Jie Li, George Michelogiannakis, Brandon Cook, John M. Shalf, Yong Chen · 2024
Disaggregated architectures that separate memory and compute resources have gained interest recently [1]. Past research has mainly examined the effects of remote memory on jobs and various implementation architectures. However, there's a gap in understanding the optimal size of local and remote memory and in developing scheduling policies to improve both application performance, job queuing time, and memory utilization. This paper focuses on three areas using a data-driven approach: examining the scale of memory disaggregation and its tradeoffs, assessing the need for unique policies for remote memory management and their effect on system performance, and investigating job scheduling policies for specific goals in a memory-disaggregated HPC system.