Periodic activation function for fast on-line EKFtraining and pruning
Chang, Kwok‐Wo Wong, Chi-Sing Leung · Electronics Letters · 1998
In the use of the local extended Kalman filter algorithm for network training and pruning, the authors propose replacing the monotonical increasing sigmoid activation function with a periodic function. Computer simulations show that the rate of convergence is substantially improved and that more insignificant weights can be pruned.