Accelerating Distributed Matrix Multiplication with 4-Dimensional Polynomial Codes
Roy Nissim, Oded Schwartz · Society for Industrial and Applied Mathematics eBooks · 2023
A single straggler worker may delay an entire distributed system. The state-of-the-art strategies for mitigating delays in large-scale distributed matrix multiplication are polynomial-based coded computations such as the Polynomial Codes and Entangled Polynomial Codes. While such strategies deal with stragglers efficiently, they discard partial computations performed by stragglers. Hence, they are sub-optimal. Here, we present the Multi Entangled Polynomial Codes, a straggler mitigation strategy that utilizes the computations performed by all workers and significantly reduces the running time. Furthermore, it allows the final output to be decoded before any worker completes its tasks, thereby breaking the lower bound of Yu, Maddah-Ali, and Avestimehr (2020). Previous studies that utilize partial computations performed by stragglers require large Maximal Distance Separable codes, resulting in high overhead costs. In contrast, our strategy requires short codes comparable to Entangled Polynomial Codes. Thus, we preserve efficient encoding and decoding complexity and reduce the arithmetic overhead of previous solutions by a factor of where N and W are the matrices dimension and the number of workers, respectively. We provide experimental results on an Amazon EC2 cluster that demonstrate up to 15% speedup over previous strategies. Moreover, we show that our strategy is optimal up to a factor of (1 + o(l)).