Enhanced Q -Learning Approach to Finite-Time Reachability With Maximum Probability for Probabilistic Boolean Control Networks
Hongyue Fan, Jingjie Ni, Fangfei Li · IEEE Transactions on Control of Network Systems · 2025
In this paper, we investigate the problem of controlling probabilistic Boolean control networks (PBCNs) to achieve maximum probability reachability within a finite time horizon. We address three key questions: 1) How to find control policies that achieve maximum probability of reachability within a finite time horizon, especially when time follows a specific distribution, 2) How to achieve faster convergence in solving the first question when adjusting the finite time setting as compared to training from scratch, and 3) How to efficiently address these issues for large-scale PBCNs. For question 1), we demonstrate the applicability ofQ-learning (QL) method on the finite-time reachability problem. For question 2), in light of the potential changes in the finite time setting, we incorporate transfer learning (TL) technique to leverage prior knowledge and accelerate convergence speed. For question 3), we propose an enhanced model-freeQL approach that effectively addresses these challenges in large-scale PBCNs by introducing memory-efficient modifications, thus improving upon the traditionalQL algorithm. Finally, we apply the proposed method to two examples: a small-scale PBCN and a large-scale PBCN, demonstrating the effectiveness of our approach.