Automated Machine Learning for Enhanced Software Reliability Growth Modeling: A Comparative Analysis with Traditional SRGMs
Taehyoun Kim, Duksan Ryu, Jongmoon Baik · 2024
Traditional Software Reliability Growth Models (SRGMs) depend on unrealistic assumptions, which makes it difficult to capture the complexities of modern software development. Recent advancements in artificial intelligence have introduced new modeling techniques, but these methods are complex and require careful algorithm selection and hyperparameter tuning. Automated Machine Learning (AutoML) has emerged as a promising solution to streamline this process. However, its application in the field of software reliability growth modeling remains unexplored. In this study, we explore the effectiveness of AutoML in enhancing software reliability growth modeling and compare its performance with traditional SRGMs. We employ two prominent AutoML packages, Auto-sklearn and H2O AutoML, and leverage twelve project datasets to answer three research questions: (1) the impact of various AutoML package options on software reliability growth modeling, (2) the identification of the optimal AutoML approach for modeling software reliability growth, and (3) the overall effectiveness of AutoML in software reliability growth modeling. We found that using ensemble options enhanced predictive performance across multiple projects. Auto-sklearn with the ensemble option emerged as the most effective approach when evaluated based on End-point Prediction values, while H2O AutoML with the ensemble option demonstrated superior performance based on Mean Squared Error values. Additionally, AutoML packages with ensemble options demonstrated more accurate predictive performance compared to traditional SRGMs across the majority of datasets. Our study highlights the potential of AutoML to enhance software reliability growth modeling and provides insights for future research and practical applications in software engineering.