Dynamic Batch Scheduler for Efficient Neural Networks Training
Mutayyba Waheed, Jinlong Li · 2023
The widespread utilization of neural networks (NNs) in recent years has been driven by their remarkable ability to discern complex data patterns. However, training NN models on vast datasets can be computationally expensive. To address this challenge, we investigate the art of data batch scheduling, which involves strategically organizing extensive datasets into smaller, manageable batches for training. This paper presents a novel dynamic batch scheduling algorithm explicitly designed for training deep neural networks. It dynamically constructs batches based on a combination of training loss and batch utilization, leading to optimized training efficiency. The experiments conducted on MNIST, CIFAR-10, and CIFAR-100 datasets demonstrate the superiority of our approach over recognized methods. Our dynamic batch scheduler, when combined with different, models, optimizers, batch sizes, numbers of batches per epoch, and time complexity, consistently improves training and testing accuracy while maintaining algorithmic efficiency.