Probing with Precision: Probing Question Generation for Architectural Information Elicitation
Gokul Rejithkumar, Preethu Rose Anish, Jyoti Shukla, Smita S. Ghaisas · 2024
Software Requirements Specifications (SRS) often lack the necessary level of specificity required by software architects to make well-informed architectural decisions. This deficiency compels software architects to probe business analysts to collect more details pertinent to architectural requirements from the clients. In our previous work, we introduced Probing Question-flows (PQ-flows) that can assist business analysts to probe stakeholders and gather architecturally significant information for the creation of a more comprehensive SRS. Key limitations of our previous work were the manually created templatized PQ-flows and the mapping of PQ-flows to the software requirements based on standard Vector Space Model. In this study, we propose a Retrieval Augmented Generation (RAG) prompting framework to address these limitations. We conducted experiments using ChatGPT and Mistral-7B models. We present our findings utilizing human and automated evaluation metrics on a subset of the publicly available PUblic REquirements (PURE) dataset.