Momentum Based on Adaptive Bold Driver
Shengdong Li, Xueqiang Lv · 2019
The momentum-based stacked attention networks (SANs) is one of the best models for image question answering. However, we find that it is easy to fall into the local optimal solution, which results in the higher question answering error rate. To solve the problem, we propose adaptive bold driver (ABD). The experimental results and analysis show that it outperforms the state-of-the-art global learning rate adaptive algorithm in the local learning rate adaptive stochastic gradient descent (SGD). It is deeply integrated with momentum, and we propose momentum based on ABD (MABD). The experimental results show that its accuracy is 2.33% higher than the baseline (momentum), 2.54% higher than momentum based on bold driver, and 1.80% higher than the annealing-based momentum. The experimental analysis proves that it is the state-of-the-art optimization algorithm in the SANs-based image question answering and it has effectiveness, significance, generalization performance, and promotional value.