Identifying Impactful Agents Via Faux Adversarial Games
Carmel Fiscko, Soummya Kar, Bruno Sinopoli · 2022
Identifying important agents is a key goal in study of multi-agent systems (MAS), but standard graph-based centrality metrics often fail to account for the long-term influence of an agent through a system. In this work we model a MAS as a controlled Markov decision process (MDP), and quantify an agent's impact as its maximum ability to minimize the MDP's value function. To evaluate this metric, the agent is considered as a “faux adversary” and a secondary MDP is defined from their viewpoint. A game is defined between the controller of the main MDP and the adversary of the secondary MDP, who each work to optimize their respective policies. We show that defining the Q functions of the MDPs as the players' utilities results in a zero-sum game whose Nash equilibrium value is equal to the proposed impact metric; the exact value can then be found using traditional methods in game theory. Finally, simulations demonstrate that the proposed impact metric can be effective in reinforcement learning settings, and shows how simulated game-play can yield the Nash equilibrium value.