Advanced Methodologies for Optimal Neural Network Design and Performance Enhancement
Sreelekha Paul, Sanchita Ghosh, Dipanjan Das, Saptarshi Kumar Sarkar · Advances in computer and electrical engineering book series · 2024
The overview also includes state of the art methods for enhancing and optimizing the architectural structures of neural networks for machine learning and AI. Several optimization strategies are used in training deep learning models; they include Gradient Based Approaches, Learning Rate Schedules, Architectural Enhancements, and the incorporation of new architectures such as Transformers Recurrent Neural Networks, Residual Networks, and Convolution Neural Networks. Some of the other areas dealt with are development of new hardware accelerators, aspects of computationally efficient algorithm, and various forms of regularization. Also discussed are transfer learning, meta-learning, and changing hyperparameters. The development of the new neural network architectures for the future is a significant endeavor and includes the following concepts: Integration of Neural Networks with Quantum Computing, Federated Learning, Neuromorphic Computer Systems, Symbolic Reasoning. Better, faster, and more accurate systems, expected in the future as the neural network optimization techniques are improved.