Design Methodology for an Automated Penetration Testing System Based on Improved Monte Carlo Tree Search
Yishuo Wang, Chaobin Huo, Shaojie Wang, Minchao He, Yiting Liu · 2023
This paper presents a design methodology for an automated penetration testing system, drawing inspiration from the AlphaGo concept. In this paper, an improved Monte Carlo tree search method (MCTS) is used, which combines the traditional MCTS algorithm with the trained deep neural network to guide the node expansion, so as to achieve faster and better attack path determination. In the lateral movement process of penetration testing, the system uses the improved MCTS algorithm to select the next action in the search space according to the current environment state and known vulnerability information. Firstly, the DQN algorithm is used to train and learn in the environment, and then the trained deep Q-network is used to provide more reliable decision-making guidance in the search process, so as to speed up the speed of finding the optimal attack path. The experimental results show that the automated penetration testing system proposed in this paper has achieved significant improvement in the path planning task in the post-penetration testing stage.