Stratified Cross-Validation on Multiple Columns
Jan Motl, Pavel Kordík · 2021 IEEE 33rd International Conference on Tools with Artificial Intelligence (ICTAI) · 2021
Stratified cross-validation is one of the standard methods of how to evaluate classifier’s generalization accuracy. However, conventional implementations of cross-validation can stratify only by a single column. In this paper, we propose to utilize Integer Linear Programming in order to enable stratification by multiple columns. Our experiments using an extensive set of multi-label data sets shows that the proposed method significantly outperforms non-stratified cross-validation.