Automated Writing Trait Analysis
Paul Deane · 2024
Writing quality comprises multiple dimensions of rhetorical, conceptual, linguistic, and orthographic structure and reflects multiple skills and strategies. Evidence of quality can be gathered either from product or process features. With respect to the written product, what counts as quality varies depending on genre and purpose. While automated writing evaluation (AWE) models use features reflecting these factors, and sometimes explicitly identify factor structures, they are not explicit multidimensional theories of text structure. Historically, such models emerged from genre analysis. There is much to be gained from building explicitly multidimensional AWE models, including the ability to support formative assessment, address genre differences, and profile writing development. Evidence from a keystroke log can also be used to capture and profile features of the writing process. Process evidence is predictive of essay quality, but provides a way to track progress on foundational skills, including keyboarding, and makes it possible to measure the extent to which writers self-regulate the writing process by switching strategically among planning, drafting, editing, and revision activities. This chapter presents a multidimensional AWE model, trained on more than 1.37 million student submissions to a digital classroom writing tool, that combines product and process features to create rich profiles of student writing that can be used to profile student strengths and weaknesses, track student progress, and differentially measure response to intervention on multiple dimensions of writing quality.