Trans-cache: a caching mechanism for LSM-tree based on filtering and transformation
Yuxin Zhang · 2024
In the field of large-scale data processing, databases built on Log-Structured Merge Trees (LSM-trees) have gained wide acceptance due to their superior write performance. However, their relative weaker read performance, especially when dealing with massive disk I/O operations and complex cache management, remains a significant drawback. Existing block cache and KV cache strategies alleviate this issue to some extent, but challenges still persist in terms of cache invalidation, memory space wastage, and inconsistent optimization effects for different queries. This study proposes and implements a novel cache structure capable of conditionally converting block cache into KV cache, thus enabling more precise caching of frequently point queried key-value pairs. This not only significantly enhances point query performance, but also reduces wastage of memory space, while mitigating cache invalidation issues caused by merge operations. Our experiments show that using the same size of memory, the new strategy resulted in a 14% performance improvement for point queries compared to the original LevelDB.