HF-DQN: Human in the Loop Reinforcement Learning Methods with Human Feedback for Autonomous Robot Navigation
Shaofan Wang, Ke Li, Tao Zhang, Zhao Zhang, Zhenning Hu · 2024
Reinforcement learning agents can learn to solve sequential decision tasks by interacting with the environment. However, as the complexity of the state space and task increases, the exploration effort required by these agents grows exponentially. To address this limitation, human knowledge can be integrated as valuable supplementary information for the agent. One effective method is imitation learning, where the agent learns by mimicking human-demonstrated decisions. However, human guidance need not be limited to demonstrations. In some applications, expert demonstration data may not be available, and other forms of guidance may be more appropriate, requiring less human effort. The first contribution of this work is the proposal of a concise human feedback-based reinforcement learning (HF -DQN) algorithm. This method incorporates human feedback to aid the RL process, providing guidance without requiring full demonstrations. Secondly, we constructed multiple simulated environments for autonomous navigation tasks, including ego-vehicle obstacle avoidance, visual obstacle avoidance, and UAV landing, to evaluate various TAMER (Training an Agent Manually via Evaluative Reinforcement) framework-based methods. Additionally, standard dueling-DQN was also implemented for comparison. Our findings show that HF-DQN agents demonstrate stable performance and outperform their baselines for various tasks in simulated environments.