NPU-aware Parameter Generation in Quantization ONNX Model
Soyoung Lee, Minyong Sung, Jong-Hee Park, Sang-Seol Lee, Sung‐Joon Jang · 2021
In this paper, we address the design of the parameter generation method for the autonomous controlling of Neural Processing Unit (NPU) on the edge device. The design is based on quantization Open Neural Network Exchange (ONNX) model that can utilize deep learning models of various frameworks. In order to generate parameters of the quantization deep neural network (DNN) model, we parse model and generate NPU-aware parameters. When parsing model, we extract and arrange model information as parameter by traversing the quantization ONNX model. Furthermore, according to the requirements of the target NPU, the number of parameters can be reduced by offline calculation, and the floating-point arithmetic is converted into an integer arithmetic. As a result, the proposed design allows the target NPU to process various framework models by and generates NPU-aware parameters according to the target NPU. Furthermore, it can reduce the quantization parameters by average 35.4% without accuracy degradation.