An Open-Source Tool for Analyzing the Time Efficiency of Machine Learning on Edge Devices
Heba Khdr, Yiğit Oğuz, Jörg Henkel · 2024
Edge machine learning (EdgeML) refers to the practice of executing machine learning algorithms on computing devices that are close to the originating sources of data. The primary aim of EdgeML is to minimize response times and preserve data privacy. However, edge devices usually face constraints in computational power, memory, and energy. This poses difficulties for executing training and inference of intricate machine learning (ML) models, such as neural networks (NNs), on these devices. As a response, many efforts have been made to improve time efficiency11In the context of this paper, we quantify the time efficiency of an ML model by measuring the training/inference time of the given model on the target hardware. of EdgeML by optimizing ML models as well as employing hardware accelerators. Orthogonal to these endeavors, this paper presents a new open-source tool22https://github.com/TheMa3str0/EdgeMLProfiler, EdgeMLProfiler, designed to assess the time efficiency of training/inference of specified NNs, accounting for various hardware architectures and software libraries. Using our tool, we present for the first time a comparative analysis for several NN models, including well-known convolutional neural networks (CNNs) and customized Fully- connected Neural Networks (FNNs). This analysis shows varied time-efficiency trends across the tested models and hardware architectures, thereby allowing the selection of the most efficient deployment for an ML model on edge devices.