Applying the Many-Facet Rasch Model to Detect Centrality in Self-Assessment, Peer-Assessment and Teacher assessment
Farahman Farrokhi, Rajab Esfandiari, Mehdi Vaez-Dalili · 2011
Abstract: In rater-mediated second language performance assessments, where human judgment is needed to evaluate the quality of a student’s response to a particular task on a set of criteria, raters are error-prone. These unwanted, rater-dependent systematic sources of variability, which are unrelated to the students’ abilities and manifested in various ways, could endanger the fairness and validity of decisions made based on the assigned ratings. Centrality or central tendency effect is one of the most pervasive sources of rater variability which has been under-researched in EFL classroom-based assessment settings. The present study aims to employ a many-facet Rasch model (MFRM), a relatively newly developed measurement model, to detect central tendency effect in self-assessment, peer-assessment and teacher assessment. To that end, 194 assessors-188 self-and peer-assessors and 6 teacher assessors-were employed to assess 188 essays written by Iranian English majors at universities in Iran, using a six-point analytic rating scale. The data were collected and analyzed, using Facets 3.68.1. The results of Facets analysis showed that the three types of assessor-self-assessor, peer-assessor and teacher assessor-did not exhibit any sign of centrality either at group level or at individual level. The findings are promising in that they provide evidence for the viability of both self-assessors and peer-assessors for rating purposes along with teachers. Key words: Rating scale Centrality Performance assessment Self-assessment Peer assessment