The Most Generative Maximum Margin Bayesian Networks
Robert Peharz, Sebastian Tschiatschek, Franz Pernkopf · Cambridge University Engineering Department Publications Database · 2013
*These authors contributed equally to this paper Althoughdiscriminativelearningingraphical models generally improves classification results, the generative semantics of the model are compromised. In this paper, we introduce a novel approach of hybrid generativediscriminative learning for Bayesian networks. We use an SVM-type large margin formulation for discriminative training, introducing a likelihood-weighted ℓ 1-norm for theSVM-norm-penalization. Thissimultaneouslyoptimizesthedatalikelihoodandtherefore partly maintains the generative character of the model. For many network structures,ourmethodcanbeformulatedasaconvex problem, guaranteeingaglobally optimal solution. Intermsofclassification,theresulting models outperform state-of-the art generative and discriminative learning methods for Bayesian networks, and are comparable with linear and kernelized SVMs. Furthermore, the models achieve likelihoods close to the maximum likelihood solution and show robust behavior in classification experiments with missing features. 1.