Autonomous Cyber Defense using Graph Attention Network Enhanced Reinforcement Learning
Yihang Shi, Huajun Zhang, Lin Shi, Shoukun Xu · 2024
Ubiquitous networked hosts and Internet of Things (IoT) devices have enabled critical network applications in both enterprise and industrial environments. However, network threats have proliferated, constantly challenging normal network operations and data security of IoT devices. In facing these challenges, defenders have increasingly adopted Deep Reinforcement Learning (DRL) approaches, aiming to leverage their self-learning and adaptability to bolster network security. Despite these efforts, existing approaches often exhibit significant performance bottlenecks when navigating the complexities of network scenarios. This paper introduces a novel algorithm, the Normalized Graph-Attention Proximal Policy Optimization (NGA-PPO), which synergistically integrates Graph Attention Networks (GAT) with the existing Proximal Policy Optimization (PPO) framework. By harnessing a weighted mechanism to amalgamate intricate network connectivity data, NGA-PPO empowers intelligent defense agents to dynamically decipher dependencies and interactions within complex network architectures, thereby facilitating precise and prompt defense actions. Comprehensive experiments within the Yawning Titan network security simulation platform reveal that NGA-PPO outperforms other methods, delivering substantial improvements in performance and exhibiting distinct advantages in robustness and generalization capabilities. Specifically, as network scenarios become more complex, existing algorithms face significant performance limitations, whereas NGA-PPO consistently demonstrates high performance, thereby affirming its efficacy and viability in mitigating complex network threats.