Fault Resilience Analysis of Quantized Deep Neural Networks
Rizwan Tariq Syed, Markus Ulbricht, Krzysztof Piotrowski, Milos D. Krstic · 2021
Because of the ground-breaking results in many different fields, Deep Neural Networks (DNNs) have also been widely adopted for safety-critical applications i.e. autonomous driving and space exploration. These applications are not only resource-constrained but also require a high level of reliability. DNNs quantization is viewed as a viable method to minimize the implementation cost and maximize the resiliency of the DNNs while preserving the inference accuracy. We perform a comprehensive layer-wise fault analysis of homogeneous and heterogeneous quantized DNNs and study the impact of faults (e.g., soft errors modeled as bit flips) in the DNNs' weights. The findings of this study suggest that quantizing the DNN model to fewer bits helps to increase the resiliency of the model i.e., quantizing the model from a fixed point (FxP) 32-bit to FxP 4-bit can increase the resiliency of the model by 20.7 % at 10 % Bit Error Rate (BER). But more vigorous quantization could sacrifice the resiliency and accuracy.