An Optimal Error Correction Scheme for the Shuffle Phase of a MapReduce Distributed Computing System
Anjana A. Mahesh, Nujoom Sageer Karat, Balaji Sundar Rajan · IEEE Communications Letters · 2021
The MapReduce model of distributed computation accomplishes a task in three phases - two computation phases-Map and Reduce, with a communication phase - Shuffle, happening in between. This letter looks at the distributed communication problem in the shuffle phase with the assumption that the links through which the computing nodes exchange information is error-prone. Under this assumption, an optimal linear error-correcting transmission scheme is designed using index coding techniques.