A Short Introduction to Model Selection, Kolmogorov Complexity and Minimum Description Length (MDL)
Volker Nannen · arXiv (Cornell University) · 2010
The concept of overfitting in model selection is explained and demonstrated with an example. After providing some background information on information theory and Kolmogorov complexity, we provide a short explanation of Minimum Description Length and error minimization. We conclude with a discussion of the typical features of overfitting in model selection.