Predicting Software System Quality Indicators Using Modifications of the SHAP Method
Anton Shantyr · Visnyk of Vinnytsia Politechnical Institute · 2024
This paper proposes five suggestions for improving the practical application of the SHAP method in the context of software system (SS) quality assessment. The study explores the potential for enhancing the SHAP method (Shapley Additive Explanations) in forecasting the quality indicators of software systems. The aim of the research is to increase the accuracy and adaptability of the SHAP method through modifications that account for various SS quality parameters, including performance, reliability, scalability, and usability. The objectives of the study are the following: to conduct a detailed review of the issues associated with the application of the SHAP method in SS quality assessment; to mathematically describe five methods for modifying the SHAP method to improve its accuracy, adaptability, and speed in evaluating SS quality indicators; and to experimentally verify the proposed modifications to assess their effectiveness compared to the original method. The study examines five methods for enhancing the SHAP method: LSTM (Long Short-Term Memory); CNN (Convolutional Neural Networks); adaptive SHAP; MLP (Multilayer Perceptron); and RNN (Recurrent Neural Network, including model ensembles and Bayesian updating). A practical comparison of the results demonstrated that the proposed variations of SHAP could significantly improve accuracy and prediction speed, especially in dynamic conditions and large data volumes. The development and justification of the five optimization modifications of the SHAP method achieved improvements in accuracy and forecasting efficiency for SS quality indicators. The research found that methods using deep neural networks (LSTM, CNN) show higher accuracy and adaptability compared to the original SHAP; however, this comes at the cost of implementation complexity and longer execution times. Adaptive SHAP and Ensemble are optimal in terms of balancing accuracy, adaptability, and interpretability, but require some optimization to improve execution time. The original SHAP showed satisfactory accuracy (MAE 0.84) and interpretability (7/10), but lagged behind modern approaches in adaptability (6/10) and execution time (0.4 hours). SHAP needs further optimization, especially in dynamic environments where rapid adaptation to new data is crucial. Time series (LSTM) provided the highest accuracy among the methods (MAE 0.91) and good adaptability (8/10).