Asynchronous and Adaptive Checkpoint for WAL-based Data Storage Systems
Li Zhu, Yanpeng Hu, Chundong Wang · 2023
Write-ahead logging (WAL) is widely utilized to ensure data’s integrity for data storage systems. Modified data is firstly written to a WAL file. Then data is persistently flushed to original home location for in-place update. These two steps are referred to as commit and checkpoint. In this paper, we take SQLite in the WAL mode to study the impact of checkpoint. Once 1,000 pages accumulate in the WAL file, SQLite checkpoints them to the database file with fsync. Such a periodical checkpoint fashion causes substantial spikes to the user-facing latency of inserting or updating data over time. Also, the fixed checkpoint frequency of every 1,000 pages does not consider the runtime write/read access pattern. We propose an algorithm named Walack. Walack conducts fsync asynchronously for each checkpoint. By observing write and read requests, it online adjusts the checkpoint frequency. These two strategies jointly enable Walack to gain both high performance and space efficiency. Experiments show that Walack reduces the user-facing tail latency by up to 92.3% for write requests, with both average write and read performances retained.