Deep Learning for Resource-Limited Devices
Mohammed Amine Merzoug, Ahmed Mostefaoui, Mohammed H. Kechout, Sebti Tamraoui · 2020
In recent years, deep neural networks have revolutionized the development of intelligent systems and applications in many areas. Despite their numerous advantages and potentials, these intelligent models still suffer from several issues. Among them, the fact that they became very complex with millions of parameters. That is, requiring more resources and time, and being unsuitable for small restricted devices. To contribute in this direction, this paper presents (1) some state-of-the-art lightweight architectures that were specifically designed for small-sized devices, and (2) some recent solutions that have been proposed to optimize/compress classical deep neural networks to allow their deployment on embedded systems. In this paper, we also present our four-staged approach that aims to further enhance the performance of lightweight object-detection models. The conducted performance evaluations have demonstrated that the proposed four layers (basic model initialization, pruning, quantization, and adaptation) considerably reduce the redundancy of filters and model size with negligible performance degradation.