Family and Prejudice: A Behavioural Taxonomy of Machine Learning Techniques
Raül Fabra-Boluda, Ferri Cèsar, Fernando Martínez‐Plumed, José Hernández‐Orallo, Marïa José Ramírez-Quintana · Frontiers in artificial intelligence and applications · 2020
One classical way of characterising the rich range of machine learning techniques is by defining 'families', according to their formulation and learning strategy (e.g., neural networks, Bayesian methods, etc.).However, this taxonomy of learning techniques does not consider the extent to which models built with techniques from the same or different family agree on their outputs, especially when their predictions have to extrapolate in sparse zones where insufficient training data was available.In this paper we present a new taxonomy of machine learning techniques for classification, where families are clustered according to their degree of (dis)agreement in behaviour considering both dense and sparse zones, using Cohen's kappa statistic.To this end, we use a representative collection of datasets and learning techniques.We finally validate the taxonomy by performing a number of experiments for technique selection.We show that ranking techniques by only following prejudice -the reputation they have for other problems-is worse than selecting techniques based on family diversity.