Partially Labeled Data Tuple Can Optimize Multivariate Performance Measures
Jim Jing-Yan Wang, Xin D. Gao · 2015
Multivariate performance measure optimization refers to learning predictive models such that a desired complex performance measure can be optimized over a training set, such as the F1 score. Up to now, all the existing multivariate performance measure optimization methods are limited to a completely labeled data tuple, i.e., the label tuple is complete. However, in real-world applications, sometimes it is difficult to obtain a complete label tuple. In this paper, we show that the multivariate performance measures can also be optimized by learning from partially labeled data tuple, when the label tuple is incomplete. We introduce a slack label tuple to represent the sought complete true label tuple, and learn it jointly with a hyper predictor, so that it can be consistent to the known labels, prediction results, and is smooth in the neighborhood. We develop an iterative learning algorithm to learn the slack label tuple and the hyper predictor. Its advantage over state-of-the-art multivariate performance measure optimization methods is shown by experiments on benchmark data sets.