Mean-field approximation minimizes relative entropy
Griff L. Bilbro, Wesley E. Snyder, Reinhold C. Mann · Journal of the Optical Society of America A · 1991
We derive the mean-field approximation from the information-theoretic principle of minimum relative entropy instead of by minimizing Peierls’s inequality for the Weiss free energy of statistical physics theory. We show that information theory leads to our statistical mechanics procedure. As an example, we consider a problem in binary image restoration. We find that mean-field annealing compares favorably with the stochastic approach.