Deep Reinforcement Learning in Maximum Entropy Framework with Automatic Adjustment of Mixed Temperature Parameters for Path Planning
Yingying Jennifer Chen, Fengkang Ying, Xiangjian Li, Huashan Liu · 2023
Deep reinforcement learning in maximum entropy framework is sample-efficient and has a strong exploration capacity, making it effective and favorable to solve problems like path planning. Properly tuning the temperature parameters can improve the performance of policy learning, but manual tuning is inefficient. In this paper, we propose a mixed algorithm named SAC-M which is inspired by adaptive soft actor-critic (A-SAC) and soft actor-critic with automatic entropy (SAC-A). The proposed method achieves automatic adjustment of temperature parameters so that the entropy can vary among different states to control the degree of exploration, reducing the possibility of learning suboptimal policies to some extent. The experimental results illustrate that the proposed SAC-M outperforms A-SAC and SAC-A in path planning tasks in different scenes, especially when A-SAC and SAC-A are mixed in a proper ratio.