Bayesian Network Classifiers: Model Approximation and Learning Complexity
Ashutosh Garg, Dan Roth · 2002
Bayesian Networks are becoming increasingly popular in the machine learning community %, not just for inference but also for as a classification tool. When learning BN classifiers, due to lack of knowledge of true model, one makes a number of assumptions which lead to {\em modeling error}. The parameters of this model are further estimated from data, contributing to {\em estimation errors}. This paper analyzes both kinds of errors. We first bound the modeling error in terms of KL-distance between the true and the approximate distribution, and provide indications that ``often'''' this error is negligible. We then study the {\em learning complexity}, and bound the number of examples needed to minimize the estimation error in terms of the data {\em margin}.