A Hybrid Asymmetric Integer-Only Quantization Method of Neural Networks for Efficient Inference
Wei Dong Lu, Zhong Ma, Yang Chaojie · 2022 5th International Conference on Pattern Recognition and Artificial Intelligence (PRAI) · 2022
The demand for adopting neural networks in resource-constrained embedded devices is continuously increasing. Quantization is one of the most promising solutions to reduce computational cost and memory storage on embedded devices. In order to reduce the complexity of deploying neural networks on Integer-only hardware, most of the current quantization methods use a symmetric quantization mapping strategy. However, the robustness and generalization of this quantization mapping strategy are poor. It is difficult for the quantized model to meet the accuracy requirements, especially for complicated tasks with higher accuracy requirements. In this paper, an efficient hybrid asymmetric Integer-only quantization method for different types of neural network layers is proposed. The proposed method can resolve the contradiction between the quantization accuracy and the ease of implementation, and balance the trade-off between clipping range and quantization resolution, and thus improve the accuracy of the quantized neural network. The results show that compared with the traditional symmetric quantization method, the accuracy of the proposed method can be improved up to 2.02% for classification models and up to 5.52% for the Yolo-v3 tiny target detection model, enabling the neural network to be easily deployed and implemented on embedded devices.