Alps: Adaptive Quantization of Deep Neural Networks with GeneraLized PositS
Hamed F. Langroudi, Vedant Karia, Zachariah Carmichael, Abdullah M. Zyarah, Tej Pandit, John Leroy Gustafson, Dhireesha Kudithipudi · 2021
In this paper, a new adaptive quantization algorithm for generalized posit format is presented, to optimally represent the dynamic range and distribution of deep neural network parameters. Adaptation is achieved by minimizing the intra-layer posit quantization error with a compander. The efficacy of the proposed quantization algorithm is studied within a new low-precision framework, ALPS, on ResNet-50 and EfficientNet models for classification tasks. Results assert that the accuracy and energy dissipation of low-precision DNNs using generalized posits outperform other well-known numerical formats, including standard posits.