A study on a low power optimization algorithm for an edge-AI device
T. Kaneko, Kentaro Orimo, Itaru Hida, Shinya Takamaeda-Yamazaki, Masayuki Ikebe, Masato Motomura, Tetsuya Asai · Nonlinear Theory and Its Applications IEICE · 2019
Although research on the inference phase of edge artificial intelligence (AI) has made considerable improvement, the required training phase remains an unsolved problem. Neural network (NN) processing has two phases: inference and training. In the training phase, a NN incurs high calculation cost. The number of bits (bitwidth) in the training phase is several orders of magnitude larger than that in the inference phase. Training algorithms, optimized to software, are not appropriate for training hardware-oriented NNs. Therefore, we propose a new training algorithm for edge AI: backpropagation (BP) using a ternarized gradient. This ternarized backpropagation (TBP) provides a balance between calculation cost and performance. Empirical results demonstrate that in a two-class classification task, TBP works well in practice and compares favorably with 16-bit BP (Fixed-BP).