Approaches to the computerized assessment of free text responses
D. Whittington, Harry B. Hunt · Loughborough University Institutional Repository (Loughborough University) · 1999
The automated assessment of student's essays is regarded by many as the Holy Grail of computer aided assessment.Whilst a few people search for the grail, many more deny its existence.This paper describes the various approaches that have been taken over the last 40 years in an attempt to solve the problems involved with the computerized assessment of free text.The earliest approaches were founded in simple style analysis.Systems, such as Project Essay Grade (PEG), were developed upon the idea that certain surface features of an essay could be manipulated in such a way as to predict the grade that a human examiner would assign to an essay.Other methods, such as Latent Semantic Analysis (LSA), also take a statistical approach to marking, but focus on actual textual content, analyzing groupings and context.The Educational Testing Service (ETS) originally attempted to tackle the problem from a classification point of view whilst more recent work by ETS bears similarities to PEG in its statistical approach.Most of the methods currently being developed have been shown to be capable of generating essay scores that correlate with a human grader's scores at least as well as two human graders correlate with each other..A novel approach being adopted by the authors to allow the comparison of students' essays against a model answer involves the use of theories developed for interlingual machine translation.A few different methods for knowledge representation, and their current uses in machine translation are presented.Panlingua, an idea for knowledge representation developed by Chaumont Devin, is based on semantic network research.Using a system of nodes over four layers it attempts to model how the brain might translate from sensory patterns it sees or hears at the top level, through syntactic and semantic levels, to a representation of understanding at the deepest level.Another approach to machine translation involves work done by Bonnie Dorr at the University of Maryland based on Lexical Conceptual Structure (LCS) theory.This allows the knowledge represented in a text to be translated into a language independent data structure.These theories provide a way of representing knowledge that is not reliant on the surface syntax of the text representing the knowledge.This will hopefully allow 'fuzzy matching' of sentences which have different syntactic structures but similar semantic meaning.The authors will be looking at the possibility of grading essays via a comparison of these data structures.