Answer-state Recurrent Relational Network (AsRRN) for Constructed Response Assessment and Feedback Grouping
Zhaohui Li, Susan Lloyd, Matthew Beckman, Rebecca J. Passonneau · 2023
STEM educators must trade off the ease of assessing selected response (SR) questions, like multiple choice, with constructed response (CR) questions, where students articulate their own reasoning.Our work addresses a CR type new to NLP but common in college STEM, consisting of multiple questions per context.To relate the context, the questions, the reference responses, and students' answers, we developed an Answer-state Recurrent Relational Network (AsRRN).In recurrent time-steps, relation vectors are learned for specific dependencies in a computational graph, where the nodes encode the distinct types of text input.AsRRN incorporates contrastive loss for better representation learning, which improves performance and supports student feedback.AsRRN was developed on a new dataset of 6,532 student responses to three, two-part CR questions.AsRRN outperforms classifiers based on LLMs, a previous relational network for CR questions, another graph neural network baseline, and few-shot learning with GPT-3.5.Ablation studies show the distinct contributions of AsRRN's dependency structure, the number of time steps in the recurrence, and the contrastive loss.