Accurate Estimation of the CNN Inference Cost for TinyML Devices
Thomas Garbay, Khalil Hachicha, Petr Dobiáš, Wilfried Dron, Pedro Lusich, Imane Khalis, Andréa Pinna, Bertrand Granado · 2022
Our society will be deeply impacted by neural network inference on embedded devices. Many of them are based on the use of microcontroller units (MCUs) which are extremely resource-scarce. The best modality to solve most of computer vision problems are artificial intelligence algorithms such as Convolutional Neural Networks (CNNs). Although, a CNN’s accuracy implies significant inference costs within the targeted hardware: an important energy consumption, a high latency, and a significant memory footprint. This is the Tiny Machine Learning (TinyML) paradigm that aims at bringing machine learning inference to ultra-low-power devices. Therefore, estimating this inference cost within a given microcontroller unit is a design key. To this end, we introduce an estimation method for the energy consumption, the latency, and the memory space required by a CNN within MCUs. The CNNs investigated are LeNet5 and ResNet8 within several TinyML devices. One inference cost for both neural networks is estimated in a range of 14 frequencies. The proposed method shows an average estimation error of 6.91% on energy consumption, 8.79% on latency, and 3.93% on the required memory space.