A metric for unsupervised metalearning
Jun Won Lee, Christophe G. Giraud-Carrier · Intelligent Data Analysis · 2011
We argue the value of unsupervised metalearning and discuss the attendant necessity of suitable similarity, or distance, functions. We leverage the notion of diversity among learners used in ensemble learning to design a distance function for the clustering of learning algorithms. We revisit the mo st popular measures of diversity and show that only one of them, Classifier Output Difference (COD) is a metric. We then use COD to produce a clustering of 21 learning algorithms, and show how this clustering differs from a clustering based on accuracy, and how it can be used to highlight interesting, sometimes unexpected, similarities among learning algorithms.