Striking the Right Chord: Parameter Tuning in Memory Tiering Systems
Konstantinos Kanellis, Sujay Yadalam, Shivaram Venkataraman, Michael M. Swift · 2025
Memory tiering achieves cost-effective scaling by adding multiple tiers of memory using new interconnect technologies, such as CXL, or denser alternatives, such as NVM. For maximum performance, frequently accessed (hot) data must be placed close to the CPU in faster tiers, while infrequently accessed (cold) data can be placed in farther, slower tiers. Existing tiering solutions employ heuristics and preconfigured thresholds to make data placement and migration decisions. As a result, these systems fail to adapt to different workloads and hardware, resulting in poor performance. In this paper, we study how the thresholds and parameters used by tiering systems affect application performance. We find that applications are sensitive to these values, and that the optimal values vary significantly across workloads. Furthermore, factors such as application inputs, thread counts, fast-slow memory ratio, and memory latency and bandwidth influence the best parameter choices. Our evaluation shows that properly configuring these parameters can improve performance by up to 2x compared to default configurations.