Rule-Based Methods for Automated Scoring: Application in a Licensing Context
2006
It is fitting that this chapter follows one on human scoring since a rule-based approach to scoring essentially attempts to embody the reasoning of experts, conceived as a combination of logical evaluations. It is also appropriate that it appears early in the volume because, in comparison to some of the methods discussed in subsequent chapters, logical rules are conceptually straightforward and one of the earliest approaches to representing expert decision-making. A byproduct of this conceptual simplicity is that the relationship between programmed rules and outcomes are directly inspectable. In this chapter, we demonstrate how relatively simple rules can be harnessed to execute the equivalent of what would generally be considered higher order reasoning. In this case, the higher order reasoning involves the expert evaluation of solutions to complex constructed response tasks, submitted as part of a nationwide licensure examination.