On the Limits of Parallelizing Convolutional Neural Networks on GPUs
Behnam Pourghassemi, Chenghao Zhang, Joo Hwan Lee, Aparna M. Chandramowlishwaran · 2020
GPUs are currently the platform of choice for training neural networks. However, training a deep neural network (DNN) is a time-consuming process even on GPUs because of the massive number of parameters that have to be learned. As a result, accelerating DNN training has been an area of significant research in the last couple of years.