Normalized Post-training Quantization for Photonic Neural Networks
Manos Kirtas, Nikolaos Passalis, Athina Oikonomou, George Mourgias-Alexandris, Miltiadis Moralis‐Pegios, Nikos Pleros, Anastasios Tefas · 2022 IEEE Symposium Series on Computational Intelligence (SSCI) · 2022
The recent advances of Deep Learning (DL) have fueled the research of hardware accelerators, which can signif-icantly improve the computation speed and energy efficiency. Neuromorphic photonics are among the most promising emerging approaches, as they can operate at very high frequencies with minimal energy and power consumption. At the same time, such analog architectures also impose a number of limitations arising from noise sources, digital-to-analog (DAC), and analog-to-digital (ADC) conversions that lead to reducing the effective bit resolution for DL models. The main contribution of this work is a photonic-compliant post-training quantization method that effectively allows us to deploy limited precision models without significant performance degradation, enabling us to reduce the cost and complexity of photonic substrates. The proposed method is capable of effectively handling outliers and equally distributing the quantized parameters, improving the accuracy of DL models even when compared with more computationally-intensive methods, such as quantization-aware training approaches. The effectiveness of the proposed method is demonstrated in Convolutional Neural Networks (CNNs), employing different photonic configurations, lowering the models' precision requirements down to 4-bits.