Practical and Robust Safety Guarantees for Advanced Counterfactual Learning to Rank
Shashank Gupta, Harrie Oosterhuis, Maarten de Rijke · 2024
Counterfactual learning to rank (CLTR) can be risky and, in various circumstances, can produce sub-optimal models that hurt performance when deployed. Safe CLTR was introduced to mitigate these risks when using inverse propensity scoring to correct for position bias. However, the existing safety measure for CLTR is not applicable to state-of-the-art CLTR methods, cannot handle trust bias, and relies on specific assumptions about user behavior.