Approximate Bayesian Neural Network Trained with Ensemble Kalman Filter
Chao Chen, Xiao Lin, Yuan Huang, Gabriel Terejanu · 2019
Neural networks have achieved significant success in many areas. Nevertheless, conventional neural networks lack uncertainty information, which plays an important role especially in critical-safety applications such as self-driving cars. When uncertainty is characterized using probability the main modeling approach is the construction of Bayesian neural networks. Obtaining the posterior distribution for these models is computationally intensive and analytical solutions are intractable. In this work, we propose a novel algorithm to infer the weights for Bayesian neural networks based on the ensemble Kalman filter. To evaluate the performance of the algorithm, we use ten regression datasets from University of California at Irvine machine learning repository, and a natural language dataset. The results suggest that EnKF can be used as a gradient-free alternative to training deep neural networks to capture prediction uncertainty.