Low-Latency Caching for Cloud-Based Web Applications
Adam Wolfe Gordon, Paul Lu · 2011
Many Web applications are now hosted in elastic cloud en-vironments where the unit of resource allocation is a virtual machine (VM) instance; entire VMs are added or removed to scale up or scale down. A variety of techniques can reduce the latency of communication between VMs co-located on the same server in, say, a private cloud. For example, par-avirtualized network mechanisms (e.g., vhost and virtio in Linux KVM) can optimize the number of protection bound-ary crossings. Inter-VM shared memory can further reduce boundary crossings after setting up a shared region. We present the design, implementation, and an evalua-tion of Nahanni memcached, a port of the well-known mem-cached that uses inter-VM shared memory instead of a vir-tual network for cache reads. As a widely deployed cache for back-end datastores and databases, memcached’s latency is important to the performance of many well-known web sites (e.g., Facebook, Twitter) and cloud platforms (e.g., Google’s App Engine). Although using shared-memory IPC is a well-known strategy, the recent introduction of the ivsh-mem inter-VM shared memory mechanism (also known as Nahanni) to Linux KVM makes the strategy practical for virtual machines. Using the Yahoo Cloud Serving Bench-mark, we confirm the intuition that Nahanni memcached can reduce the latency of cache read operations by up to 86%, and that given reasonable hit rates, this can reduce the total latency of read-related operations for a workload by up to 45 % compared to standard memcached. When using the experimental paravirtualized vhost networking mechanism in Linux KVM, Nahanni memcached offers a smaller, but still significant, advantage of 29%.