Temporal and Spatial Memory-Based Deep Reinforcement Learning for Autonomous Navigation of UAV in Complex Environments
Botong Yang, Hongyan Qian · 2023
This paper presents a memory and deep reinforcement learning (DRL) based approach for UAV navigation and obstacle avoidance, equipped with a front depth camera for observations. Earlier conventional algorithms and SLAM methods have struggled to perform well in large-scale environments, while RL based reactive obstacle avoidance suffers from local minimum problems in complex environments. Memory-based methods have achieved satisfactory results in complex environments, but existing memory methods often require large size time step and large memory capacity, making model training slow and difficult to converge. We adopt a method of mapping temporal sequences to spatial sequences, uniformly sampling observation and location information from past flight trajectories to construct a time memory structure. We then construct a spatial memory structure to store the areas that the UAV has flown over, as a supplement to the time memory, encouraging the UAV to explore new areas. The model based on recurrent neural networks and attention mechanisms is used to train UAV flight strategies in complex environments and compared with other memory-based approaches. Experiments show that our method outperforms other methods in different environments and exhibits stronger robustness in new environments. Finally, we confirm the significance of our proposed temporal and spatial memory structures through ablation experiments.