Network-Constrained P2P Trading: A Safety-Aware Decentralized Multi-Agent Reinforcement Learning Approach
Qianyi Ma, Zifa Liu, Yujian Ye, Xiao Liu · IEEE Transactions on Smart Grid · 2025
With increasingly predominant renewable energy sources and market deregulation in the distribution network, considerable research effort has been devoted to safe reinforcement learning (RL). Network-constrained peer-to-peer (P2P) transaction is particularly challenging, as the system-wide global constraint is implicitly determined by the joint behavior of individual agents. Existing safe RL methods are burdened of addressing this type of constraint. In this context, this paper proposes a decentralized Multi-Agent Safety-Aware Learning (MASAL) method based on trust region concept. It treats system operation safety as a prerequisite and realizes safety-aware target. Furthermore, a system-wide safety condition classification is proposed, categorizing trading policy optimization into five operation conditions. Endogenous and exogenous violations can be distinguished. Building on the classification of system operation conditions, the method conducts preventive trading policy under normal conditions, and responsive policy under critical conditions when exogenous violations occur. Case studies show that the proposed method outperforms state-of-the-art safety layer method and Lagrangian relaxation method under normal and critical conditions. Numerical results also highlight the effectiveness of the method in terms of interpretable trading policies, scalability performance, and training robustness with multiple constraints.