Transformer Neural Network-based Transfer Learning for Economic Dispatch of Microgrids
Dhruv Singh Kushwaha, Zoleikha Abdollahi Biron, Miaomiao Hu · 2024
Reinforcement learning (RL) algorithms have proven to be effective in computing control policies for complex dynamical systems. However, like most data-driven optimal control approaches, sample efficiency and high computational complexities create problems in practical uses. Transfer learning has proven to be effective in mitigating these problems by utilizing pre-trained RL agents for similar tasks. Whilst transfer learning can speed up training times for environments with similar action and state spaces, it still requires re-training the RL agent in the new environment. In this work we propose a transformer neural network (TNN)-based transfer learning approach for RL, where re-training of RL agent in the new environment is circumvented, giving a transferable policy for multiple use cases. We consider a scenario of transferring a trained policy from a small microgrid to a large microgrid and vice-versa. The policy is transferred by training a TNN to map the state space from source to the target environment. Transferred policy for economic dispatch in each case is compared to a baseline quadratic programming (QP) solution and policy for a RL agent exclusively trained for that scenario. Results show identical performance considering cost of operation as the metric, in transferring the policy and an agent exclusively trained on the microgrid.