From Baselines to DenseNet: A Deep Learning Framework for CNN Optimization and Augmentation
Hazim Shatnawi, Mahmoud Abusaqer, Jamil M. Saquer · 2025
This paper presents a systematic investigation into the interplay among convolutional neural network (CNN) architectures, optimization algorithms, and data augmentation strategies. Our study examines three widely used architectures---Baseline CNN, ResNet-50, and DenseNet-121---across four datasets representing distinct domains: CIFAR-10, FashionMNIST, PathMNIST (from MedMNIST), and Street View House Numbers. We evaluated four optimizers (SGD, Adam, RMSprop, and AdamW) under both augmented and non-augmented conditions over 100 training epochs, measuring performance in terms of validation accuracy, loss, and execution time. Our experiments reveal that optimal configurations vary by dataset: DenseNet-121 with AdamW demonstrates exceptional performance on CIFAR-10 and SVHN, DenseNet-121 with Adam achieves the highest accuracy on FashionMNIST, and the Baseline CNN with Adam is most efficient on PathMNIST. These findings underscore the importance of dataset-specific tuning and provide actionable insights for balancing model complexity, computational cost, and predictive performance. Our study thus serves as a practical guide for both academic research and real-world applications.