Deterministic Policy Gradient with Advantage Function for Fixed Wing UAV Automatic Landing
Xin Yuan, Yuewen Sun, Yuanda Wang, Changyin Sun · 2019
This paper addresses the autolanding problem for fixed wing unmanned aerial vehicles (UAV) in the presence of the downburst. The proposed method is developed based on the reinforcement learning methodology. The solution consists of a path-tracking controller for the glideslope maneuver and an attitude controller for the flare maneuver. Both controllers are designed in continuous state and action spaces. In our study, two complementary techniques are proposed within the framework of deterministic policy gradient (DPG). First, the advantage function is introduced in the critic network to improve the performance of the learning process. The proposed representation of the action value function consists of two parts: the low-frequency part and the high-frequency one. Second, a two-stream network is developed to tackle with the issue of partially observable Markov decision process (POMDP). The architecture synthesizes past experiences to perform policy evaluations and policy improvements, which has successfully improved the robustness of the learned policy. The described performance of our approach is illustrated in flight simulations under the influence of the wind field.