Scaling Exponential Backoff
Michael A. Bender, Jeremy T. Fineman, Seth Lewis Gilbert, Maxwell Young · Journal of the ACM · 2018
Randomized exponential backoff is a widely deployed technique for coordinating access to a shared resource. A good backoff protocol should, arguably, satisfy three natural properties: (1) it should provide constant throughput, wasting as little time as possible; (2) it should require few failed access attempts, minimizing the amount of wasted effort; and (3) it should be robust, continuing to work efficiently even if some of the access attempts fail for spurious reasons. Unfortunately, exponential backoff has some well-known limitations in two of these areas: it can suffer subconstant throughput under bursty traffic, and it is not robust to adversarial disruption. The goal of this article is to “fix” exponential backoff by making it scalable, particularly focusing on the case where processes arrive in an online, worst-case fashion. We present a relatively simple backoff protocol, R e -B ackoff , that has, at its heart, a version of exponential backoff. It guarantees expected constant throughput with dynamic process arrivals and requires only an expected polylogarithmic number of access attempts per process. R e -B ackoff is also robust to periods where the shared resource is unavailable for a period of time. If it is unavailable for D time slots, R e -B ackoff provides the following guarantees. For n packets, the expected number of access attempts for successfully sending a packet is O (log 2 ( n + D )). For the case of an infinite number of packets, we provide a similar result in terms of the maximum number of processes that are ever in the system concurrently.