Multiple mini-robots navigation using a collaborative multiagent reinforcement learning framework
Piyabhum Chaysri, Konstantinos Blekas, Kostas I. Vlachos · Advanced Robotics · 2020
Piyabhum Chaysri, Konstantinos Blekas* & Kostas Vlachos Department of Computer Science and Engineering, University of Ioannina, Ioannina, GreecePiyabhum Chaysri received the B.Sc. degree in Computer Science in 2016 and received the M.Sc. degree in Technologies and Applications in 2018, both from the University of Ioannina, Ioannina, Greece. He currently continues his Ph.D. research in Reinforcement Learning Agents in Robotic Applications at the same university. His research interests include machine learning, intelligent agents, reinforcement learning, navigation and control of robotic platforms as well as designing and construction of robotic mechanisms.Konstantinos Blekas received the Diploma degree in Electrical Engineering in 1993 and the Ph.D. degree in Electrical and Computer Engineering in 1997, both from the National Technical University of Athens. He is currently associate professor at the Department of Computer Science & Engineering, University of Ioannina, Greece. He has co-authored more than 90 refereed journal and conference articles. His research interests include machine learning, pattern recognition, intelligent agents and reinforcement learning with applications to robotics and autonomous systems, computer vision, and medicine.Kostas Vlachos received the B.Sc. degree in electrical engineering from the Technical University of Dresden, Dresden, Germany, in 1993. He received from the National Technical University of Athens, Athens, Greece, the M.S. (2000) and Ph.D. (2004) degrees in the area of Automatic Control and Robotics respectively. From 1996 to 1998, he worked as a Software Analyst in INTRACOM S.A. From 2007 to 2013, he was a visiting Lecturer at the Mechanical Engineering Department, University of Thessaly, where he taught courses in the areas of Control Systems, and Robotics. Currently, he is an Assistant Professor with the Department of Computer Science and Engineering, University of Ioannina. Research interests: haptic mechanisms, medical simulation, microrobotics, navigation and control of robotic mechanisms.CONTACT Konstantinos Blekas [email protected] Department of Computer Science and Engineering, University of Ioannina, Ioannina, 45110 GreeceIn this work we investigate the use of a reinforcement learning (RL) framework for the autonomous navigation of a group of mini-robots in a multi-agent collaborative environment. Each mini-robot is driven by inertial forces provided by two vibration motors that are controlled by a simple and efficient low-level speed controller. The action of the RL agent is the direction of each mini-robot, and it is based on the position of each mini-robot, the distance between them and the sign of the distance gradient between each mini-robot and the nearest one. Each mini-robot is considered a moving obstacle that must be avoided by the others. We propose suitable state space and reward function that result in an efficient collaborative RL framework. The classical and the double Q-learning algorithms are employed, where the latter is considered to learn optimal policies of mini-robots that offers more stable and reliable learning process. A simulation environment is created, using the ROS framework, that include a group of four mini-robots. The dynamic model of each mini-robot and of the vibration motors is also included. Several application scenarios are simulated and the results are presented to demonstrate the performance of the proposed approach.GRAPHICAL ABSTRACT