Soft-Actor-Attention-Critic Based on Unknown Agent Action Prediction for Multi-Agent Collaborative Confrontation
Ziwei Liu, Changzhen Qiu, Zhiyong Zhang · 2023
Reinforcement learning in multi-agent scenarios is very important for practical applications, but the challenges in this scenario are far greater than those in the single-agent environment. In order to solve the multi-agent scene problems, we propose an algorithm called soft-actor-attention-critic. The algorithm adopts centralized training with distributed execution, and adds a centralized critic. The critic selects the relevant information of each agent at each time step through the attention mechanism, and makes decisions based on this information. At the same time, the counterfactual baseline is introduced to the credit allocation problem, and the agents are trained accordingly. In addition, a supervised learning method based on GRU(gated recurrent unit) is proposed to predict the behavior of unknown agents. In the complex multi-agent cooperation environment, this method has higher learning efficiency than the latest method. Our method is not only applicable to collaboration scenarios with shared rewards, but also to confrontation scenarios with personalized rewards, as well as scenarios without global state. Therefore, it has enough flexibility and can be applied to most multi-agent learning problems.