Security Requirements Classification by Means of Explainable Transformer Models
Luca Petrillo, Fabio Martinelli, Antonella Santone, Francesco Mercaldo · 2025
Security requirements are important in the identity management domain since they help protect sensitive personal and organizational data from unauthorized access and breaches. However, manual classification of these requirements is time-consuming and error-prone. This paper proposes a domain-independent automated classification method of security requirements based on transformers. The proposed approach uses four pre-trained transformer models: BERT, RoBERTa, and two distilled versions called DistilBERT and DistilRoBERTa. Through a hyperparameter tuning phase, we identify the two best-performing models. Following, using the best hyperparameter configuration, we fine-tuned these models, achieving an accuracy value of 0.89 with BERT and an F1-Score of 0.92 with RoBERTa. Finally, we provided explainability of the fine-tuned transformers via SHAP analysis, which helps interpret the model predictions by assigning importance scores to words. This approach helps identify which terms impact the security classification more and provides a clearer insight into the reasoning behind the model's decisions.