Deep reinforcement learning-based DoS attack and its countermeasures in cyber-physical systems
Mengyu Huang · 2022
With the rapid development and widespread use of wireless communication technologies, the security issue in cyber-physical systems (CPSs) has attracted much attention. Due to the exposure and vulnerability of wireless communication networks, an external attacker can easily cause severe damage to infrastructure systems. An important benchmark of CPS security is the cyber attack designs. Among various types of attacks, the denial-of-service (DoS) attack is one of the most common and achievable attacks in real applications. However, most previous works related to the DoS attack design assume that the attacker has complete knowledge of the CPS, which may be unrealistic. In addition, the existing methods cannot be easily extended to a large-scale multi-process CPS because of the curse of dimensionality. Therefore, there is a strong need to propose an attack design that is not only applicable to attackers with limited information, but also scales to large-dimensional systems. In this thesis, we fill this research gap by taking advantage of deep reinforcement learning (DRL) and present several DRL-based DoS attack designs and countermeasures in a variety of scenarios. First, we study the discrete DoS attack power design and propose a double deep Q-network (DDQN)-based attack power allocation. To further improve data efficiency, inspired by model-based RL, we introduce two enhanced attack algorithms with auxiliary tasks of transition estimation. Second, we consider the continuous attack power design and propose a deep deterministic policy gradient (DDPG)-based attack power allocation. To deal with the coupling power constraint in multi-process systems, we provide an extension version with a feasibility layer. Attack strategics in such continuous space could greatly outperform solutions established in a discrete space. Third, we introduce a hierarchical framework of DoS attack design to integrate the tasks of attack channel selection and attack power allocation. We propose a D2 attack algorithm and its improved version with a self-attention mechanism to accelerate the learning process. This hierarchical learning-based attack design provides a general architecture that can be easily adapted to different cases. Finally, we discuss potential countermeasures against the DRL-based DoS attack, which can help to improve the reliability and robustness of different CPSs.