Tracing Feature Tests to Textual Requirements
Mahima Dahiya, Mark Junjie Li, Glen Horton, Thomas Scherz, Nan Niu · 2024
Software features deliver values to the end users, and thus their qualities shall be assured. While the mainstream quality assurance technique is software testing, its adequacy can only be assessed by tracing the testing artifacts to system requirements. In this paper, we report an in-depth case study aiming to trace all the feature tests of a Web application to textual requirements. To that end, we experiment two automated trace recovery methods: vector space model and transformer-based semantic embedding. The tracing results, unfortunately, are not satisfactory. We then explore the use of large language models, OpenAI’s ChatGPT in particular. We engage the original developers into evaluating the tracing and ChatGPT-prompting results. Our study reveals the promises of exploiting large language models to assist in software tracing activities.