On hyper-parameter estimation in empirical Bayes: a revisit of the MacKay algorithm
Chun’e Li, Yongyi Mao, Richong Zhang, Jinpeng Huai · Uncertainty in Artificial Intelligence · 2016
An iterative procedure introduced in MacKay's evidence framework is often used for estimating the hyper-parameter in empirical Bayes. Despite its effectiveness, the procedure has stayed primarily as a heuristic to date. This paper formally investigates the mathematical nature of this procedure and justifies it as a well-principled algorithm framework. This framework, which we call the MacKay algorithm, is shown to be closely related to the EM algorithm under certain Gaussian assumption.