Avoiding Pitfalls in Networked Key-Value Store for Tiered Memory
Seungmin Shin, L. Felmingham Kim, Wookyung Lee, Eyee Hyun Nam, Seungmin Kim, Bryan Suk Joon Kim, Sung-Jin Lee, Eunji Lee · 2025
This paper describes the performance pitfalls when using tiered memory for a networked key-value store and our approach to avoiding them. We observe that when receiving data over the network, writing data to tiered memory results in multiple data stagings and repetitive user-kernel crossings. We also observe sudden bursts of I/O operations when allocating memory if the slower memory tier is backed by a DAX file system. We address these challenges through (1) PPF (packet peek and forward) that peeks at the packets in the kernel layer with eBPF and streamlines data placement decisions on tiered memory, and (2) OMA (opportune memory allocator) that moves zeroing off the critical path. Performance evaluation with the prototype shows that the adoption of our design improves IOPS by up to 128%.