Minimum Description Length Model Selection
Steven de Rooij · UvA-DARE (University of Amsterdam) · 2008
When I took prof.Vitányi's course on Kolmogorov complexity, I was delighted by his seemingly endless supply of anecdotes, opinions and wisdoms that ranged the whole spectrum from politically incorrect jokes to profound philosophical insights.At the time I was working on my master's thesis on data compression with Breanndán Ó Nualláin, which may have primed me for the course; at any rate, I did well enough and Paul Vitányi and Peter Grünwald (who taught a couple of guest lectures) singled me out as a suitable PhD candidate, which is how I got the job.This was a very lucky turn of events for me, because I was then, and am still, very much attracted to that strange mix of computer science, artificial intelligence, information theory and statistics that make up machine learning and learning theory.While Paul's commonsense suggestions and remarks kept me steady, Peter has performed the job of PhD supervision to perfection: besides pointing me towards interesting research topics, teaching me to write better papers, and introducing me to several conferences, we also enjoyed many interesting discussions and conversations, as often about work as not, and he said the right things at the right times to keep my spirits up.He even helped me think about my future career and dropped my name here and there. . .In short, I wish to thank Paul and Peter for their excellent support.Some time into my research I was joined by fellow PhD students Tim van Erven and Wouter Koolen.I know both of them from during college when I did some work as a lab assistant; Wouter and I in fact worked together on the Kolmogorov complexity course.The three of us are now good friends and we have had a lot of fun working together on the material that now makes up a large part of this thesis.Thanks to Wouter and Tim for all their help; hopefully we will keep working together at least occasionally in the future.Thanks also go to Daniel Navarro, who made my stay in Adelaide very enjoyable and with whom I still hope to work out the human psyche one day, and to my group members Rudi Cilibrasi, Lukasz D ֒ ebowski, Peter Harremoës and Alfonso Martinez.Regards also xi to Harry Buhrman, and the people of his excellent quantum computing group.In addition to my supervisors, I wish to thank the following people for making technical contributions to this thesis.Chapter 2 is inspired by the work of Aaron D. Lanterman.Thanks to Mark Herbster for a lively discussion and useful comments with respect to Chapter 4, and to Tim van Erven for his excellent suggestions.