Robust RegBayes: Selectively Incorporating First-Order Logic Domain Knowledge into Bayesian Models
Shike Mei, Jun Zhu, Junwei Zhu · 2014
Much research in Bayesian modeling has been done to elicit a prior distribution that incorpo-rates domain knowledge. We present a novel and more direct approach by imposing First-Order Logic (FOL) rules on the posterior distribution. Our approach unifies FOL and Bayesian model-ing under the regularized Bayesian framework. In addition, our approach automatically estimates the uncertainty of FOL rules when they are pro-duced by humans, so that reliable rules are incor-porated while unreliable ones are ignored. We apply our approach to latent topic modeling tasks and demonstrate that by combining FOL knowl-edge and Bayesian modeling, we both improve the task performance and discover more struc-tured latent representations in unsupervised and supervised learning. 1.