Evaluation of Natural Language Processing for Requirements Traceability
Christopher D. Laliberte, Ronald E. Giachetti, Mathias Kölsch · 2022
Requirements traceability remains a challenge, especially in multi-level system of systems being developed by many different organizations. This paper develops and tests automated tracing methods based on Natural Language Processing (NLP) techniques to help ensure links between parent and child requirements are correct while preventing common requirements traceability issues. Using publicly available requirements documentation from the National Aeronautics and Space Administration (NASA), the developed software tool analyzed 215 requirements, generated a Term Frequency – Inverse Document Frequency (TF-IDF) matrix of the document collection, and classified parent-child requirement pairs using the histogram distance and cosine similarity measures under eighteen different similarity measure thresholds. Precision, recall, and F-scores were calculated, yielding maximum F-scores for each similarity measure with the objective of understanding the performance and utility of histogram distance for automated requirements tracing. The results indicate natural language processing is likely not a practical approach to requirements traceability.