PowerGridworld: A Framework for Multi-Agent Reinforcement Learning in Power Systems [SWR-22-07]
David Biagioni, Rohit H. Chintala, Xiangyu Zhang, Ahmed S. Zamzam, Jennifer King, Deepthi Vaidhynathan, Dylan Wald · OSTI OAI (U.S. Department of Energy Office of Scientific and Technical Information) · 2021
NREL's PowerGridworld provides a modular simulation environment for training heterogenous, grid-aware, multi-agent reinforcement learning (RL) policies at scale. The package enables the user to create component gym environments that can be composed into more complex agents. For example, a grid interactive building environment can be created by composing together component environments each encapsulating the building, PV, and battery physics. These multi-component environments can then be combined into multi-agent simulation where each agent's power consumption/injection becomes an input for solving the optimal power flow on a distribution feeder modeled in OpenDSS. Information from OpenDSS, such as bus voltages and line flows, can be included in the agents' observation spaces to enable grid-aware rewards. The default API for the PowerGridworld simulator conforms to RLLib's MultiAgent API and thus enables distributed training using HPC and cloud resources.