Enhancing Deep Learning Optimizers for Detecting Malware Using Line Search Method under Strong Wolfe Conditions
Mohammed A. Saleh · 2023
Optimizers play a crucial role in deep learning neural networks (DL-NNs). An optimizer is an algorithm that adjusts the parameters of deep learning neural networks (DL-NNs) to improve their performance and increase their precision. An optimizer relies heavily on a learning rate, known as tweaking hyper-parameter, which states the iteration step size while heading toward the loss function minimization. In fact, the choice of learning rate is tricky, since a low value of learning rate ends in a lengthy training process that gets a cumbersome, while a high value of learning rate leads to the selection of an inadequate weight set or an unsustainable training process. This paper proposes a novel enhanced deep learning approach called Deepictor, which aims to enhance convergence performance of the deep learning (DL) throughout prior learning rates computation using line search method under strong Wolfe conditions, which have been proven for enhancing convergence performance.