What is Next when Sequential Prediction Meets Implicitly Hard Interaction?
Kaixi Hu, Lin Li, Qing Jie Xie, Jianquan Liu, Xiaohui Tao · 2021
Hard interaction learning between source sequences and their next targets is challenging, which exists in a myriad of sequential prediction tasks. During the training process, most existing methods focus on explicitly hard interactions caused by wrong responses. However, a model might conduct correct responses by capturing a subset of learnable patterns, which results in implicitly hard interactions with some unlearned patterns. As such, its generalization performance is weakened. The problem gets more serious in sequential prediction due to the interference of substantial similar candidate targets.