Deep statistics
Tome Eftimov, Gašper Petelin, Rok Hribar, Gorjan Popovski, Urban Škvorc, Peter Korošec · 2020
The performance measures and statistical techniques selected affect the conclusions we can draw on the behavior of the algorithms. For this reason, we propose more robust performance statistics for addressing statistical and practical significance, as well as investigating the exploration and exploitation powers of stochastic optimization algorithms. They are introduced by the Deep Statistical Comparison approach and its variants. Its implementations are available as part of the DSCTool, which provides web services for robust ranking and hypothesis testing, including a proper selection of an omnibus statistical test and post-hoc tests if needed.