Separating Explorer for Task Inference Based Meta Reinforcement Learning Algorithm
Lanlan Gong, Xinghong Ling, Jiayou Lu, Jiaqin Zhou, Liang Xue · 2021
Traditional meta reinforcement learning based on task inference separates task inference with task control, but ignores the importance of exploration during task inference. The agent uses the same policy for both task exploration process and task control process, which leads to low task inference efficiency. To solve this problem, this paper proposes a task inference based meta reinforcement learning framework (Separating Explorer from Task Inference based Meta-Reinforcement Learning, SETIMRL). In this framework, an explorer agent is specially designed for task inference. The explorer takes the task exploration fully, and transits the collected data to the inference network. And the actor will adapt to the new tasks rapidly with the received inference information, which helps improve the model’s performance. Experimental results show that the proposed algorithm has better efficiency in multi-dimensions and sequential control tasks, compared to traditional meta reinforcement learning based on task inference.