A Sorted and Dynamic Graph Storage System of the Hybrid Memory Architecture
Ao Yang, Yutong Deng, Dingding Li, Yong Zhong Tang · 2024
Graph data is becoming dynamic and large-scale, demanding high-performance and large-capacity graph storage. Therefore, due to the performance approaching DRAM and the larger capacity than DRAM, persistent memory (PM) has been adopted in large-scale dynamic graph storage systems. However, existing PM-based dynamic graph storage systems have issues, especially PM write amplification caused by the unsorted data structure used to store edges. To improve this issue, we propose a PM-based dynamic graph storage system, HDGraph, using sorted data structure to store edges on DRAM-PM hybrid memory architecture. To better adapt the sorted data structure on PM, HDGraph employs edge buffering on DRAM, merging small writes to reduce write amplification in PM. Moreover, HDGraph also triggers buffer flushing based on a heat evaluating strategy to alleviate DRAM space pressure. Finally, HDGraph maintains a buffering log in PM for edge-level data consistency, enabling quick recovery after a crash. Experimental results show that HDGraph achieves 1.09× to 1.52× higher edge ingestion performance, compared with the only PM-based dynamic graph storage system XPGraph, which use unsorted data structure to store edges.