Tiered Cache-HNSW: Using Hierarchical Caching System in HNSW
Rei Masuda, Kazuma Iwamoto, Kazuaki Ando, Hitoshi Kamei · 2025
With the progress in generative AI technology, systems such as RAG (Retrieval-Augmented Generation) are increasingly used. In these systems, graph-based Approximate Nearest Neighbor Search (ANNS) is commonly used for background information retrieval. However, as the size of the dataset increases, the search speed becomes significantly slower. In this paper, we propose Tiered Cache-HNSW (TC-HNSW) that caches pre-categorized datasets in layers. TC- HNSW prioritizes caching frequently accessed nodes into faster storage based on the number of accesses and their category, allowing faster searches. This paper describes TC- HNSW system design, caching strategy, and the efficiency of categorization-based caching. Our preliminary evaluation shows that categorization-based caching improves performance, achieving search speeds four times faster than traditional methods when search topics are specified.