Reinforcement Learning Based on Multi-subnet Clusters
Xiaobing Wang, Gang Liu · 2019
The main task of reinforcement learning is to enable the subject to obtain the most reward from the environment. Reinforcement learning has been proposed and achieved certain results. However, many reinforcement learning methods still have inefficiencies that result in inability to meet the demand in some applications. Aiming at the above problems, this paper proposed a reinforcement learning algorithm based on multi-subnet cluster (MSC-RL). The proposed network consists of multiple subnet clusters and the primary storage network. Each subnet cluster is composed of multiple subnets and one sub-storage network. In the subnet cluster, multiple subnets are used to explore the solution space simultaneously and saves the searched information to the sub-storage network. At regular intervals, the subnet cluster saves the searched information to the primary storage network. In traditional reinforcement learning, there is not enough interaction between the independent subnets. Insufficient information interaction between the independent subnets can cause the algorithm to fall into local optimum. MSC-RL can exchange information searched by each subnet through the sub-storage network to realize information interaction within the subnet cluster. Each cluster uses the primary storage network for information interaction. The method enhances the information interaction between subnets and improves the ability of the algorithm to optimize. This paper uses the Atari game to verify the performance of the proposed method and compared it with some mainstream reinforcement learning methods. The experimental results show that the proposed algorithm is superior to some mainstream reinforcement learning methods in the performance of the Atari game.