Kikuchi-Bayes: Factorized Models for Approximate Classification in Closed Form
Aleks Jakulin, Irina Rish, Ivan Bratko · 2004
We propose a simple family of classification models, based on the Kikuchi approximation to free energy, that generalize upon the naı̈ve Bayesian classifier. The resulting product of potentials is not normal-ized, but for classification it is easy to perform the normalization for each instance separately, just as in naı̈ve Bayes. Our learning algorithm cre-ates the set of initial regions by including only those initial regions that provide a significant improvement to the approximation which does not include them. We observe that this algorithm outperforms other meth-ods, such as the tree-augmented naı̈ve Bayes, but that the inclusion of regions may increase the approximation error. For that reason we recom-mend separating the generalization error which arises from the mismatch between the training and test data, and approximation error which arises because of imperfect model. The approximation error was the dominant source of variation in experiments we performed on realistic data sets. 1