Autorank: A Python package for automated ranking of classifiers
Steffen Herbold · The Journal of Open Source Software · 2020
Analyses to determine differences in the central tendency, e.g., mean or median values, are an important application of statistics.Often, such comparisons must be done with paired samples, i.e., populations that are not dependent on each other.This is, for example, required if the performance different machine learning algorithms should be compared on multiple data sets.The performance measures on each data set are then the paired samples, the difference in the central tendency can be used to rank the different algorithms.This problem is not new and how such tests could be done was already described in the well-known article by Demšar (2006).