Relative Entropy Regularization for Robust Submodular Multi-Task Subset Selection
Ege Can Kaya, Abolfazl Hashemi · 2023
We propose a novel formulation for multi-task subset selection with the aim of finding a solution that is locally distributionally robust in the "neighborhood" of a reference distribution, assigning an importance score to each task in a multi-task objective, using relative entropy as a regularizer. Using duality, we show that this novel formulation is equivalent to a normalized, monotone nondecreasing submodular function, which can be optimized very efficiently with standard greedy-based methods. This approach bridges the existing gap in optimization of performance-robustness tradeoff in multi-task subset selection. We then experimentally corroborate our theoretical results, comparing it with two other algorithms focused on optimizing the performance of the worst-case task, and on directly optimizing the performance on the reference distribution itself. We conclude that solving our novel formulation produces a solution that performs well on the reference distribution, is locally distributionally robust, and is quick in terms of computation time.