Optimization of Heterogeneous Coded Distributed Computing with Nonuniform Input File Sizes
Jiasheng Liang, Siyu Zhang, Yong Deng · 2024
This work considers the optimal design of MapReduce-based coded distributed computing (CDC) with nonuniform input file sizes. We propose an efficient heterogeneous CDC (HetCDC) scheme capable of handling an arbitrary number of files of nonuniform sizes. We further jointly optimize the file placement and coded shuffling strategies of the proposed HetCDC by formulating an integer linear programming (ILP) problem. Finally, the performance of the optimized HetCDC is verified by numerical studies.