The Impact of Cascaded Optimizations in CNN Models and End-Device Deployment
Hanan Hussain, P. S. Tamizharasan · 2022
Optimization techniques are vital in deploying CNN models to resource-constrained devices like IoT or handheld devices. The classic optimization techniques like quantization, pruning, and clustering have proved their impact on model compression by trading off the overall accuracy. However, the performance of cascaded optimizations like (i) sparsity preserving clustering (PC), (ii) sparsity preserving quantization (PQ), (iii) cluster preserving quantization (CQ), and (iv) sparsity and cluster preserving quantization (PCQ) techniques remain unexplored. This paper studies the inference efficiency of CNN models when optimization techniques are performed in a cascaded way and individually. The feasibility of the optimization techniques was tested on both NVIDIA GTX1650 GPU and Raspberry pi-4 by considering five factors: performance metrics like (a) inference latency and (b) accuracy and (c) computational complexity of the model, resource utilization metrics like (d) energy consumption during the inference and (e) memory requirement to store the model. The results show that among all the models generated from the optimized techniques, the fully cascaded (PCQ) outperforms in terms of accuracy, latency and memory requirement metrics.