Semi-Decentralized Control of Multi-Robot System for Autonomous Navigation via Multi-Agent Reinforcement Learning
Youngjoon Lee, Gyeonghwan Kim, Changjoo Nam · Research Square · 2023
Abstract We consider the navigation problem of multiple mobile robots in a continuous space populated with obstacles. Conventionally, this problem is formulated as multi-robot path planning (MRPP) to generate collision-free paths for all robots. However, MRPP is computationally intractable owing to the involvement of multiple robots in the continuous space resulting in infinite states and actions. Also, the paths from MRPP algorithms need to consider kinematic constraints and dynamics of the robots, further complicating the planning problem. We propose an alternative approach to MRPP for improved run-time efficiency through multi-agent reinforcement learning (MARL) that learns to control the robots directly rather than computing the paths. Our MARL agent receives the information about the environment and generates control inputs to the wheels of the robots. Trained in a simulated environment with three robots and obstacles, each robot gathers the episode trajectories, consisting of observations, actions, rewards through exploring the environment where each robot perceives each other as dynamic obstacles. These trajectories train a homogeneous semi-decentralized policy that incorporates observations of the two nearest robots. The policy can be deployed to more robots, as each only requires the information about nearby robots. Without path computation or kinodynamic constraints, our method efficiently navigates a robot team in an end-to-end manner. Extensive experiments demonstrate its ability to generalize across environments with varying goal locations and obstacle layouts. Also, our method has shown to be capable of navigating more robots without additional training.