Exploring 8-Bit Arithmetic for Training Spiking Neural Networks
Tim J Fernandez-Hart, Tatiana Kalganova, James C. Knight · 2024
Spiking Neural Networks (SNNs) offer advantages over traditional Artificial Neural Networks (ANNs) in terms of biological plausibility, noise tolerance, and temporal processing capabilities. Additionally, SNNs can achieve significant energy efficiency when deployed on specialized neuromorphic platforms. However, the common practice of training SNNs using Back Propagation Through Time (BPTT) on GPUs before deployment is resource-intensive and hinders scalability. Although reduced precision inference with SNNs has been explored, the use of reduced precision during training remains largely unexamined. This study investigates the potential of posit arithmetic, a novel numerical format, for training SNNs on future posit-enabled accelerators. We evaluate the performance of 8-bit posit and floating-point arithmetic compared to 32-bit floating-point on two datasets. Our results show that 8-bit posits can match the performance of 32-bit floating-point arithmetic when all training components are quantised. These findings suggest that posit arithmetic could be a promising foundation for developing efficient hardware accelerators dedicated to SNN training. Such advancements are essential for reducing resource usage and enhancing energy efficiency, enabling the exploration of larger and more complex SNN architectures, and promoting their wider adoption.