Concurrent Learning of Bayesian Agents with Deep Reinforcement Learning for Operational Planning
T. Scott Brandes, Letitia W. Li, Daniel R. Clymer, Eric C. Blair, Michael K. Miller, Marco A. Pravia · 2024
There is a growing need in a wide range of industries to leverage AI to assist in creating operational plans. In this research, we present an AI approach for learning operational plan construction by concurrently learning Bayesian agents and deep reinforcement learning (DRL) agents that explore orthogonal parts of the decision-space. Within our concurrent learning, we use a hierarchical Bayesian program learning (HBPL) structure which provides a mechanism to incorporate a degree of structure to the learning process by leveraging knowledge from expert operational planners. We demonstrate that our concurrent learning approach out-performs a state-of-the-art full DRL approach in both speed and operational plan quality across a range of operational planning settings with increasing complexity of the decision-space. Moreover, we demonstrate that the gains in speed and plan utility from concurrent learning increase over the full DRL approach as decision-space complexity grows within the simulation environment. This work is applied to a specific setting, operational planning for air command and control; however, it provides a structure that is generalizable to a wide range of operational planning settings across industries. Additionally, we introduce a new architecture for HBPL that enables program learning within a reinforcement learning setting, providing an updated structure for Bayesian reinforcement learning.