Adaptive Design of Role Differentiation by Division of Reward Function in Multi-Agent Reinforcement Learning
Tadahiro Taniguchi, Kazuma Tabuchi, Tetsuo Sawaragi · SICE Journal of Control Measurement and System Integration · 2010
There are several problems which discourage an organization from achieving tasks, e.g., partial observation, credit assignment, and concurrent learning in multi-agent reinforcement learning. In many conventional approaches, each agent estimates hidden states, e.g., sensor inputs, positions, and policies of other agents, and reduces the uncertainty in the partially-observable Markov decision process (POMDP), which partially solve the multiagent reinforcement learning problem. In contrast, people reduce uncertainty in human organizations in the real world by autonomously dividing the roles played by individual agents. In a framework of reinforcement learning, roles are mainly represented by goals for individual agents. This paper presents a method for generating internal rewards from manager agents to worker agents. It also explicitly divides the roles, which enables a POMDP task for each agent to be transformed into a simple MDP task under certain conditions. Several situational experiments are also described and the validity of the proposed method is evaluated.