Exploring Model Compression Techniques for Efficient 1D CNN-Based Hand Gesture Recognition on Resource-Constrained Edge Devices
Mahdi Mnif, Salwa Sahnoun, Mahdi Djemaa, Ahmed Fakhfakh, Olfa Kanoun · 2024
Tiny Machine Learning is undergoing rapid evolution in the context of edge computing and intelligent Internet of Things (IoT) devices. This paper investigates the potential of model compression techniques for enhancing energy efficiency within IoT edge devices, as an application, hand gesture recognition based on Electrical Impedance Tomography data was used. To achieve efficient and accurate machine learning on battery-powered devices, three model compression techniques: Global pruning, Knowledge Distillation (KD), and quantization were investigated and applied on a 1D convolutional neural network. The global pruning technique resulted in a slight improvement in test data accuracy while maintaining the model size at 284.45 kB, though it increased the model’s sparsity. With KD, the model size significantly decreased to 75.042 kB, with negligible impact on accuracy. In contrast, 8-bit quantization reduced the model size to 78.434 kB, but this came at the cost of a substantial 13.75% decrease in accuracy. Each model compression technique contributes to reducing model size or increasing sparsity. This is particularly beneficial for deploying deep learning models on resource-constrained IoT devices.