An Approach to Improve Distributed Computing
Jing Jiang, Yiyun Zhong, Lingxiao Qu · 2021 2nd International Conference on Electronics, Communications and Information Technology (CECIT) · 2021
The coded distributed computing (CDC) proposed by Li et al. provides a high-efficiency method to decrease the communication load in most distributed computing frameworks. However, as there are more and more nodes, the numbers of output functions and input files in these CDC scenarios grow too fast to be applied in practice. In the course of our work, we make an attempt to significantly reduce the minimum requirement of $N$ and $Q$ , where $N$ denote the number of input files and $Q$ denote the number output functions in our proposed CDC scheme. The results indicate that the minimum requirement of $Q$ in our new CDC scheme is only a factor of the total number of computing nodes $K$ , and the minimum requirement of $N$ is far less than that of the optimal scheme provided by Li et al., while $\frac{L_{new}}{L_{Li}}\leq 1.03125$ , where $L_{Li}$ and $L_{new}$ are the communication loads of the Li scheme provided by Li et al. and our derived CDC scheme, respectively.