Optimization Techniques in Training Deep Neural Networks for Vision
Shantanu Bindewari, Sumit Singh Dhanda, Anand Mohan Singh · 2025
Computer vision has undergone a revolution because of deep neural networks (DNNs), which have enabled improvements in segmentation, object detection, and image categorization. However, appropriate optimization methods, such as hyperparameter tuning and model architecture design, are critical to these models’ success. This book examines several approaches to optimizing deep neural networks for vision applications, with an emphasis on crucial techniques that boost performance, increase training efficiency, and deal with issues including overfitting, disappearing gradients, and expensive computing. We start by going over the basics of optimization, including regularization strategies, gradient descent variations, and learning rate schedules. We cover advanced topics like as adaptive learning rate algorithms, second-order optimization strategies, and model compression for faster inference in later chapters. A particular focus is on hyperparameter tuning techniques, such as Bayesian optimization, population-based training, and traditional grid and random search. Case studies on convolutional neural networks (CNNs) optimization, object identification models, such as YOLO and Mask R-CNN, and more recent designs like vision transformers are also included in this book chapter. These case studies show how optimization approaches can be used in real-world visual tasks. Future directions in the field include automated machine learning and the use of quantum computers for optimization and role of (AutoML) and neural architecture search (NAS), are discussed in depth.