Applying fast matrix multiplication to neural networks
Ahmed Khaled, Amir F. Atiya, Ahmed H. Abdel-Gawad · 2020
Recent advances in deep neural networks have enabled impressive performance in computer vision, natural language processing, and other fields, yet they remain computationally very intensive to train or use. We consider the use of Winograd's Algorithm for fast matrix multiplication in feedforward neural networks and we find that speedups of 10% -- 30% are possible for fully connected layers in large networks.