Negative log-likelihood and statistical hypothesis testing as the basis of model selection in IDEAs
Peter A. N. Bosman, Dirk Thierens · 2000
In this paper, we analyze the most prominent features in model selection criteria that have been used so far in iterated density estimation evolutionary algorithms (IDEAs, EDAs, PMBGAs). These algorithms build probabilistic models and estimate probability densities based upon a selection of available points. We show that the negative log{likelihood is a basis of the inference features when the Kullback{Leibler divergence is used. We show how previously found to be problematic issues in the case of continuous random variables can be resolved by starting from the derived basics. By doing so, we have a probabilistic model search metric that can be justied through the use of statistical hypothesis tests. This in turn reduces the need for additional complexity penalties.