Trash day: coordinating garbage collection in distributed systems
Martin Maas, Tim Harris, Krste Asanović, John D. Kubiatowicz · 2015
Cloud systems such as Hadoop, Spark and Zookeeper are frequently written in Java or other garbage-collected lan-guages. However, GC-induced pauses can have a signifi-cant impact on these workloads. Specifically, GC pauses can reduce throughput for batch workloads, and cause high tail-latencies for interactive applications. In this paper, we show that distributed applications suffer from each node’s language runtime system mak-ing GC-related decisions independently. We first demon-strate this problem on two widely-used systems (Apache Spark and Apache Cassandra). We then propose solv-ing this problem using a Holistic Runtime System, a dis-tributed language runtime that collectively manages run-time services across multiple nodes. We present initial results to demonstrate that this Holistic GC approach is effective both in reducing the impact of GC pauses on a batch workload, and in improv-ing GC-related tail-latencies in an interactive setting. 1