Self-imitation Learning for Action Generation in Text-based Games
Zijing Shi, Yunqiu Xu, Meng Fang, Ling Chen · 2023
In this work, we study reinforcement learning (RL) in solving text-based games.We address the challenge of combinatorial action space, by proposing a confidence-based self-imitation model to generate action candidates for the RL agent.Firstly, we leverage the self-imitation learning to rank and exploit past valuable trajectories to adapt a pre-trained language model (LM) towards a target game.Then, we devise a confidence-based strategy to measure the LM's confidence with respect to a state, thus adaptively pruning the generated actions to yield a more compact set of action candidates.In multiple challenging games, our model demonstrates promising performance in comparison to the baselines.