A note on the evidence and Bayesian Occam's razor
Iain Murray, Zoubin Ghahramani · Cambridge University Engineering Department Publications Database · 2005
Abstract—In his thesis, MacKay (1991) introduced figure 1, explaining how Bayes rule provides an automatic “Occam’s razor ” effect, penalizing unnecessarily complex models. This figure has been adopted by several authors in the same schematic form. Here, after briefly reviewing necessary material, we compute a realization of the plot for a toy data modeling problem. We discuss interesting aspects of this plot and their implications for understanding model complexity. I.