Constraining Bayesian network learning with qualitative models
Eric E. Altendorf · 2005
approved: Thomas G. Dietterich Machine learning encompasses probabilistic and statistical techniques that can build models from large quantities of extensional information (examples) with minimal dependence on intensional information (domain knowledge). This focus of machine learning is reflected in the never-ending quest for off-the-shelf classifiers. To generalize to unseen data, however, we must make use of more information than is contained in the training data. Successful learning from data, especially from sparse data, relies on effective incorporation of domain knowledge into the learning algorithm. Unfortunately, there are very few existing techniques or technologies for doing this. Feature engineering, feature selection, model structure selection, parameterization, and algorithm selection are common techniques, but all of these are difficult, time-consuming, and limited in the type of knowledge they can express. In this paper, we show how to interpret qualitative knowledge about probabilistic influences, in particular, knowledge about monotonicity, synergy, and strength of influence, as constraints on conditional probability distributions. We then describe a method for incorporating this knowledge into a parameter estimation algorithm for Bayesian networks. We provide results demonstrating improved accuracy for monotonicity-constrained networks, especially with very small training sets, compared to unconstrained networks and other learning algorithms. Redacted for privacy ©Copyright by Eric E. Altendorf May 6, 2005 All Rights Reserved Constraining Bayesian Network Learning with Qualitative Models by Eric E. Altendorf