RAGCacheSim: A discrete-event simulator for evaluating caching strategies in Retrieval-Augmented Generation systems
Hardik Ruparel, Tatsat Patel · Software Impacts · 2025
Retrieval-Augmented Generation (RAG) systems enhance large language models (LLMs) with external knowledge retrieval but incur significant compute and latency costs. In distributed RAG deployments, semantically similar queries routed to different nodes — each with its own cache — can lead to redundant processing. We present RAGCacheSim , a discrete-event simulator for evaluating caching strategies such as Centralized Exact-match Cache (CEC), Independent Semantic Caches (IC), and Distributed Semantic Cache Coordination (DSC). It reports metrics like cache hit rate, average query latency, and coordination overhead. Built using SimPy , FastEmbed , and pybloom_live , it helps researchers optimize distributed RAG architectures. • RAGCacheSim is a discrete-event simulator for distributed RAG caching strategies. • Supports CEC, IC, and DSC cache architectures with configurable workloads. • Provides detailed metrics: cache hit rate, latency, and coordination overhead. • Enables reproducible benchmarking and optimization of RAG deployments. • Open-source Python tool built on SimPy, FastEmbed, and pybloom_live.