Quantization Profiler for Artificial Neural Networks
Martin Lindström, Jakob Hök · Lund University Publications Student Papers (Lund University) · 2020
We develop a software framework that is able to modify implementations of operators within any arti cial neural network (ANN).The framework is able to import a trained TensorFlow model and target a subset of its network layers, to provide them with custom operator implementations.Furthermore, the framework uses signal-to-quantization-noise ratio (SQNR) as a metric to identify potential layer implementations that are bottlenecks for prediction accuracy.With the use of the framework, we test various custom operator implementations for the MobileNetV neural-network architecture, which was developed by researchers Google.Speci cally, we carry out experiments that benchmark operators that are well adapted for low memory usage and execution time, e.g.-bit quantization, but have a potential cost in prediction accuracy.With our results, we prove that this tool can be useful for industries where running ANNs on devices with limited hardware, like mobile phones, are of interest.