Meta Reinforcement Learning with Hebbian Learning
Di Wang · 2022 IEEE 13th Annual Ubiquitous Computing, Electronics & Mobile Communication Conference (UEMCON) · 2022
Deep reinforcement learning achieves super-human performances at the cost of millions of non-optimal interactions with environments. Ideally, a well trained deep reinforcement learning algorithm should be robust to unseen cases or unseen tasks with limited data. This paper proposes a new meta reinforcement learning algorithm with the Hebbian learning algorithm. According to the neuroscientific theory, Hebbian learning mimics the learning behaviours of the human brain. Neurons are activated repeatedly, and new neural networks are built with newly formed connections. Researchers have proved that compared with deep reinforcement learning, Hebbian learning can obtain the near-optimal solutions within short training iterations. In this paper, we use a teacher-student learning architecture. In the teacher model, the Hebbian learning model is trained with an evolutionary strategy while the student model is a deep reinforcement learning model trained with modified training procedures. Experiments prove that our proposed algorithm overwhelms the performances of the Hebbian learning algorithm and the original deep reinforcement learning vastly. Besides, with strong inductive biases introduced by learned knowledge from subtasks, the data efficiency of deep reinforcement learning is significantly boosted. Moreover, we set the target task and subtasks different primarily to prove the robustness of Hebbian learning as the teacher model. Experiments prove that this proposed meta reinforcement learning algorithm could boost data efficiency largely.