Efficient accuracy estimation for instance-based incremental active learning.
Christian Limberg, Heiko Wersing, Helge Joachim Ritter · PUB – Publications at Bielefeld University (Bielefeld University) · 2018
Estimating system’s accuracy is crucial for applications of in- cremental learning. In this paper, we introduce the Distogram Estimation (DGE) approach to estimate the accuracy of instance-based classifiers. By calculating relative distances to samples it is possible to train an offline regression model, capable of predicting the classifier’s accuracy on unseen data. Our approach requires only a few supervised samples for training and can instantaneously be applied on unseen data afterwards. We evaluate our method on five benchmark data sets and for a robot object recognition task. Our algorithm clearly outperforms two baseline methods both for random and active selection of incremental training examples.