On a Relation between Maximum-Likelihood Classification and Minimum-Cross-Entropy Classification.
John E. Shore · Naval Research Lab. Report · 1983
Abstract : The report considers maximum likelihood (ML) and minimum cross entropy (MCE)classification of samples from an unknown probability density when the hypotheses comprise an exponential family. It is shown that ML and MCE lead to the same classification rule, and the result is illustrated in terms of method for estimating covariance matrices recently developed by Burg, Luenberger, and Wenger. MCE classification applies to the general case in which it cannot be assumed that the samples were generated by one of the hypothesis densities. The common use of ML in this case is technically incorrect, but the equivalence of MCE and ML provides a theoretical justification.