Optimal Communication and Control Strategies in a Cooperative Multiagent MDP Problem
Sagar Sudhakara, Dhruva Kartik, Rahul Kumar Jain, Ashutosh Nayyar · IEEE Transactions on Automatic Control · 2024
The problem of controlling cooperative multiagent systems under different models of information sharing among agents has received significant attention in the recent literature. In this article, we consider a setup where rather than committing to a fixed and nonadaptive information sharing protocol (e.g., periodic sharing or no sharing, etc.), agents can dynamically decide at each time step whether to share information with each other and incur the resulting communication cost. This setup requires a joint design of agents' communication and control strategies in order to optimize the tradeoff between communication costs and the control objective. We first show that agents can ignore a big part of their private information without compromising the system performance. We then provide a common-information-approach-based solution for the strategy optimization problem. This approach relies on constructing a fictitious partially observable markov decision process (POMDP) whose solution (obtained via a dynamic program) characterizes the optimal strategies for the agents. We extend our solution to incorporate time-varying packet-drop channels and constraints on when and how frequently agents can communicate.