Input-Dependent Edge-Cloud Mapping of Recurrent Neural Networks Inference
Daniele Jahier Pagliari, Roberta Chiaro, Yukai Chen, Sara Vinco, Enrico Macii, Massimo Poncino · 2020
Given the computational complexity of Recurrent Neural Networks (RNNs) inference, IoT and mobile devices typically offload this task to the cloud. However, the execution time and energy consumption of RNN inference strongly depends on the length of the processed input. Therefore, considering also communication costs, it may be more convenient to process short input sequences locally and only offload long ones to the cloud. In this paper, we propose a low-overhead runtime tool that performs this choice automatically. Results based on real edge and cloud devices show that our method is able to simultaneously reduce the total execution time and energy consumption of the system compared to solutions that run RNN inference fully locally or fully in the cloud.