Quantization Aware Training with Order Strategy for CNN
Jonghoon Kwak, Kyungho Kim, Sang-Seol Lee, Sung‐Joon Jang, Jong-Hee Park · 2022 IEEE International Conference on Consumer Electronics-Asia (ICCE-Asia) · 2022
In this paper, we present a novel quantization-aware training method proceeding in a specific order in deep learning network. Conventional quantization techniques generally perform all layers at the same time. However, quantization sensitivity is different for each layer. For example, a quantization error increases as the quantizing bit-widths decrease, therefore the quantized model is hard to converge in the training process. To handle the issue, we propose the layer selection module which controls the order of layers to be quantized. Specifically, each layer is sequentially quantized in the order of selection so that the quantized model can be converged effectively. In our experiments, we show the quantization results on the CIFAR-10 image dataset. We also analyze the results between the different selection methods and verify the effectiveness of our proposed method.