A Hybrid Approach for Automatic Feedback Generation in Natural Language Programming
Yue Dong Zhan, Michael S. Hsiao · 2022
Developing a natural language programming (NLPr) tool is challenging due to the complex nature of natural language (NL). NLPr can be brittle and prone to failure when faced with unknown vocabulary or new sentence structures not supported by the system. In the absence of helpful feedback, the time it takes to complete the programming task might increase, and users might be discouraged. Recognizing the necessity of clear and actionable feedback, we propose a novel hybrid automatic feedback generation system in which a formal rule-based method is combined with a data-based multi-label classification (MLC) method to predict the user’s intent and present examples of known-good ways to accomplish the intended or similar tasks. We evaluate the feedback system using two phases of end-user studies, demonstrating its usefulness in helping users troubleshoot their inputs and complete their tasks.