On the Validity of Analytic Ratings.
Robert J. Marzano · 1975
ABSTRACT The purpose of this study was to examine the reliability of the analytical method of grading essays in relation to the holistic method. It was hypothesized that the'use of the analytic method to rate college composition papers produces high rater reliability at the expense of biasing the raters and thus lowering the validity cf the grades. Six essays, all on the same topic, were used for the study. It was concluded that the analytic method of rating produces a higher reliability among rate's than does the holistic method, but that the analytic method produces a lower validity for the grades on the papers than does the holistic method. On the basis of the study, the hypothesis that the analytic method lowers rater validity by introducing rater bias was logically, but not statistically, accepted. (RB)