Strategies for Evidence Identification Through Linguistic Assessment of Textual Responses
2006
It is hardly possible to overstate the importance of textual material in assessment. Most assessments are presented in textual form, and commonly include tasks with a significant textual component, such as writing, verbal comprehension, or verbal reasoning. However, when assessments require textual responses from students-whether short sentence-length answers or full-length essays-manual scoring has been the norm, with all the issues that arise from human rather than automated scoring (cf. Bejar, chap. 3, this volume, for a discussion.) The alternative-automated scoring of free textual responses-has only recently become viable, despite the potential for computer-based scoring demonstrated almost 40 years ago in Page (1966a and b). Within the last several years, a number of automatic scoring methods have been developed with applications not only to essay scoring (cf. Shermis & Burstein, 2003), but also in a variety of other educational settings such as automatic tutoring (Graesser, Wiemer-Hastings, K., Wiemer-Hastings, P., Kreuz, & Tutoring Research Group, 1999). One of the characteristics of text that must be taken into account in any approach to text scoring is the close relationship among different aspects of an assessment model: properties of the prompt; properties of the student response; models of student knowledge and of the task the student is performing and the like. There are often very strong correspondences between particular aspects of the textual product scored and multiple features of an assessment model, so that very similar techniques may often be applied to all three, and the task of disentangling which aspect of the model is being measured directly can become fairly complex.