Hybrid Distributed and Decentralised Reinforcement Learning for Formation Control of Multi‐Robots With Obstacle Avoidance
Yaoqian Peng, Xinglong Zhang, Haibin Xie, Xin Xu · CAAI Transactions on Intelligence Technology · 2025
ABSTRACT Recently, learning‐based control for multi‐robot systems (MRS) with obstacle avoidance has received increasing attention. The goals of formation control and obstacle avoidance could be intrinsically tied. As a result, developing a safe and near‐optimal control policy with the actor‐critic structure is challenging. Therefore, a hybrid distributed and decentralised asynchronous actor‐critic reinforcement learning (Di‐De‐RL) technique is proposed to address this problem. First, we decompose the integrated formation control and collision avoidance problem into two successive ones. To solve them, we design a distributed reinforcement learning (Di‐RL) algorithm that employs a neural network‐based actor‐critic structure for formation control, and a decentralised RL (De‐RL) algorithm that incorporates a potential‐field (PF)‐based actor‐critic structure for collision avoidance. In Di‐RL, the actor‐critic pairs are trained in a distributed manner to achieve near‐optimal consensus formation control. With the trained policy of Di‐RL fixed, the PF actor‐critic pairs in De‐RL are trained in a decentralised manner for safe collision avoidance. Such an asynchronous training design of the hybrid Di‐RL and De‐RL enables weight convergence and control safety in the learning process. The simulated and real‐world experimental results demonstrate the effectiveness and enhanced performance of the approach in formation control with both static and dynamic obstacle avoidance, highlighting its advantages in resolving the conflict between the safety objective and optimal control.