Linear Domain-aware Log-scale Post-training Quantization
Sung-Rae Kim, Hyun Kim · 2021
In recent years, various convolutional neural networks (CNNs) are widely used in practical applications. These networks require a lot of hidden layers to achieve high accuracy, resulting in a significant amount of computation. Therefore, studies related to network compression to support the real-time operation of CNNs in mobile devices have been actively conducted. In particular, log-scale quantization is receiving a lot of attention because it can bring many advantages in the computation amount and power consumption by changing the multiplication operations to the addition operations in the hardware accelerator design process. However, in conventional log-scale quantization methods, the parameters (e.g., weight, activation) are quantized after applying log transformation, which results in considerable quantization loss. To address this problem, this paper proposes a new technique that minimizes the accuracy degradation due to log-scale quantization by applying log transformation after performing quantization. As a result of verifying the proposed method with Cifar-10 dataset in Resnet-56 network, we have achieved much better performance than the conventional method by reducing the error rate due to quantization to within 1%.