DESIGN AND OPTIMIZATION OF NONLINEAR ACTIVATION FUNCTIONS FOR ENHANCED NEURAL NETWORK PERFORMANCE
Archana Tomar, Harish Patidar · ShodhKosh Journal of Visual and Performing Arts · 2024
This paper presents T_Saf, an innovative hybrid activation function aimed at enhancing neural network training. T_Saf combines the benefits of Softplus and Tanh, providing improved gradient stability and convergence across various tasks. Through experimental assessments on MNIST and CIFAR-10 datasets, T_Saf outperforms traditional activation functions such as ReLU, Tanh, and Leaky ReLU in terms of accuracy, convergence stability, and training robustness. The comparative analysis highlights T_Saf’s adaptability, especially in scenarios susceptible to vanishing or exploding gradients, making it a promising candidate for deep neural network applications. These results indicate that T_Saf can be a preferred activation function in challenging training environments, contributing to the overall efficiency and reliability of neural network models.