Empirically Valid Rules for Ill-Defined Domains
Collin F. Lynch, Kevin D. Ashley · 2014
Ill-defined domains such as writing and design pose chal-lenges for automatic assessment and feedback. There is lit-tle agreement about the standards for assessing student work nor are there clear domain principles that can be used for automatic feedback and guidance. While researchers have shown some success with automatic guidance through a-priori rules and weak-theory structuring these methods are not guaranteed widespread acceptance nor is it clear that the lessons will transfer out of the tutoring context into real-world practice. In this paper we report on data mining work designed to empirically validate a-priori rules with an ex-ploratory dataset in the domain of argument diagramming and scientific writing. We show that it is possible to iden-tify diagram rules that correlate with student performance but that direct correlations can often run counter to expert assumptions and thus require deeper analysis.