Out-of-Bag Estimation of the Optimal Hyperparameter in SubBag Ensemble Method
Gai-Ying Zhang, Chunxia Zhang, Jiangshe Zhang · Communications in Statistics - Simulation and Computation · 2010
SubBag is a technique by combining bagging and random subspace methods to generate ensemble classifiers with good generalization capability. In practice, a hyperparameter K of SubBag—the number of randomly selected features to create each base classifier—should be specified beforehand. In this article, we propose to employ the out-of-bag instances to determine the optimal value of K in SubBag. The experiments conducted with some UCI real-world data sets show that the proposed method can make SubBag achieve the optimal performance in nearly all the considered cases. Meanwhile, it occupied less computational sources than cross validation procedure.