Pruning Coherent Integrated Photonic Neural Networks Using the Lottery Ticket Hypothesis
Sanmitra Banerjee, Mahdi Nikdast, Sudeep Pasricha, Krishnendu Chakrabarty · 2022
Coherent integrated photonic neural network (C-IPNN) architectures enable light-speed and ultra-low-energy accelerators for rapidly growing artificial intelligence applications. Nevertheless, C-IPNNs have a large footprint and suffer from high tuning power consumption for both training and inference. Pruning C-IPNNs using model compaction to reduce the number of weight parameters can potentially alleviate these problems. However, prior attempts at pruning singular-value-decomposition-based C-IPNNs (SC-IPNNs) have shown that very few parameters can be removed without significantly degrading the network accuracy. In this paper, we present the first hardware-aware pruning method for SC-IPNNs based on the lottery ticket hypothesis (LTH). We also discuss the challenges associated with pruning SC-IPNNs and show that, in addition to the classification accuracy, model-compaction techniques should be guided by a reliability assessment of the pruned networks. As a case study, we prune a multi-layer perceptron-based SC-IPNN with two hidden layers and show that up to 89 % of the phase angles, which correspond to weight parameters in SC-IPNNs, can be pruned with a negligible loss in accuracy (smaller than 5 %) while reducing the network (i.e., tuning) power consumption by up to 86 %. Therefore, the proposed pruning method paves the way for realizing compact and energy-efficient photonic neural networks.