A Comprehensive Evaluation of Q-Learning Based Automatic Web GUI Testing
Yujia Fan, Siyi Wang, Sinan Wang, Yepang Liu, Guoyao Wen, Qi Rong · 2023
Recently, reinforcement learning (RL) based automatic Web GUI testing techniques are gaining popularity in both academia and industry as they can enable more intelligent exploration of web applications’ states. However, the existing RL-based techniques often incorporate special features, such as DFA-guided state recovery or contextual input data generation, making the effectiveness of RL itself unclear. Moreover, while these techniques mostly employ Q-learning (QL), a model-free RL method, they were evaluated with different experimental settings, which could lead to unfair comparisons. Motivated by the two observations, we propose a generic QL-based automatic Web GUI testing framework, and conduct the first systematic evaluation, which considers four QL specific configurations, on two open-source benchmark web applications and one industrial portal website. Based on the experimental results, we discuss several findings regarding the effectiveness of QL-based automatic GUI testing. We believe that our findings can provide useful guidance to industrial practitioners and shed light on future research on leveraging RL to improve automatic Web GUI testing.