A Distributed Deep Learning System With Controlled Intermediate Representation
Yucong Xiao, Yunsheng Wang, Zhipei Huang, Fei Shen, Fei Qin · 2023
The front deployed deep learning system is a promising technology for the next generation of industrial applications, which can extract essential information from high dimension sensors. However, deep learning tasks are usually computationally intensive and cannot be deployed on resource-constrained front devices. They have to be offloaded to edge or cloud devices, forming a distributed deep learning system through the exchange of intermediate representations. Consequently, the transmission loss of the intermediate representation will significantly affect the inference performance of distributed deep learning system. One of the state-of-the-art work aims to enhance the organization of intermediate representations to resist the transmission loss and guaranteeing the system performance under constrained bandwidth. In this paper, we reviewed that can be essentialized as a ‘pruning-like’ operation. Hinted by this, we further present a novel scheme named the Controlled Intermediate Representation with Information Centralized (CRIC) that accurately controls the distribution of intermediate representation without compromising the optimal inference performance. The effectiveness of CRIC has been demonstrated through extensive representative experiments.