Evaluating the variance in convolutional neural network behavior stemming from randomness
Christopher Menart · 2020
Deep neural networks are a powerful and versatile machine learning technique with strong performance on many tasks. A large variety of neural architectures and training algorithms have been published in the past decade, each attempting to improve aspects of performance and computational cost on specific tasks. But the performance of these methods can be chaotic. Not only does the behavior of a neural network vary significantly with respect to small algorithmic changes, but the same training algorithm, run multiple times, may produce models with different performance, due to multiple stochastic aspects of the training process. Replication experiments in deep neural network design is difficult in part for this reason. We perform empirical evaluations using the canonical task of image recognition with Convolutional Neural Networks to determine what degree of variation in neural network performance is due to random chance. This has implications for network tuning as well as for the evaluation of architecture and algorithm changes.