Conditional Maximum Likelihood Estimation of Naive Bayes Probability Models Using Rational Function Growth Transform
Ciprian I. Chelba, Alex Acero · 2004
We present a method for conditional maximum likelihood estimation of Naive Bayes models that employs a well known technique relying on a generalization of the Baum-Eagon inequality from polynomi-als to rational functions. The main advantage of the procedure is that it keeps the model parameter values (probabilities) properly normalized at each iteration. We apply the model trained under the maximum likelihood and conditional maximum likelihood criteria, respectively, to a text classification problem. A simple modification of the algorithm increases the convergence speed significantly over a straightforward implementation. The model trained under the con-ditional maximum likelihood criterion achieves a relative improve-ment of 40 % in classification accuracy over its maximum likelihood counterpart on a text classification task.