A Method for Regression Testing Plan Ordering for Non-Automated Executions in Black Box Testing
Vinícius de Medeiros Hernandes, André C. P. L. F. de Carvalho, Eulanda Miranda Dos Santos, Yan Soares, Hygo Sousa de Oliveira, Adamor Barros, Ronaldo Soares, Albanita Ferreira Lima, Raoni Simões Ferreira, Gabriel Martins, Lucas Carvalho, Nicolas Assumpção, José Nascimento, Eliane Collins, Silvia M. Ascate, Mateus Souza · 2025
In this paper, we propose a method for prioritizing regression test cases based on the probability of detecting software execution failures without source code analysis. To achieve this, our method employs the SentenceBERT model to extract embeddings from textual information of development commits and test scripts. These embeddings are then used by machine learning models to predict the probability of detecting a failure. Our experiments show that the proposed method achieves results equal to or better than those of human experts in 92.52% to 94.24% of scenarios when evaluating the APFD (Average Percentage Faults Detected) metric, an overall gain of 10% in APFD mean and a potential gain of up to 6.03% in test plan prioritization counting cases.