Fast Learning for Multi-Agent with Combination of Imitation Learning and Model-Based Learning for Formation Change of Transport Robots
Keisuke Azetsu, Almira Budiyanto, Nobutomo Matsunaga · 2024
In recent years, cooperative transport using autonomous mobile robots has attracted attention in multi-agent systems which control multiple agents simultaneously. Although many cooperative transport methods have been studied, they have been limited to simple transport tasks. In particular, it cannot cope with the complex situations in which the target transport formation changes with many robots. High-speed calculation for multi-agent using reinforcement learning is required for these situations to realize flexible and efficient transformation. However, it is known that it takes time to learn the robot’s trajectory when changing the formation using reinforcement learning. For example, MADDPG, which is an Actor-Critic method, Deep-Dyna Q with a model-based algorithm and an improved model Dyna-MADDPG algorithm, which is specialized for multi-agent systems for formation, were proposed. In this paper, a GASIL-MADDPG method is proposed, which simultaneously applies model-based learning and imitation learning to MADDPG for fast learning. As a result, the learning time was 50% faster than MADDPG and 37% faster than Dyna-MADDPG by using the proposed method.