Examining the Invariance of Rater and Project Calibrations Using a Multi-facet Rasch Model.
Thomas R. O’Neill, Mary E. Lunz · 1996
To generalize test results beyond the particular test administration, an examinee's ability estimate must be independent of the particular items attempted, and the item difficulty calibrations must be independent of the particular sample of people attempting the items. This stability is a key concept of the Rasch model, a latent trait model of probabilities that permits items and persons to be analyzed independently, yet still be compared using a common frame of reference. An extension of the Rasch model, the multi-facet Rasch model, can estimate examinee ability, item difficulty, and other facets for polytomous data. It was hypothesized that the multi-facet Rasch model would yield invariant, sample-free, slide and judge calibrations for a certification test for histology completed by 364 candidates. Eighteen qualified judges graded the test, which required examinees to prepare laboratory slides. Results of the study confirm that the slide and judge calibrations were essentially stable across diverse samples of examinees. This indicates that slide and judge calibrations can be used to anchor test administrations to a benchmark scale, making the equating of two administrations of the examination possible and supporting the hypothesis. (Contains 2 figures, 2 tables, and 15 references.) (SLD) *********************************************************************** * Reproductions supplied by EDRS are the best that can be made * * from the original document. * *********************************************************************** U.S. DEPARTMENT OF EDUCATION Orrice I Educahonal Research end Improvement EDU ATIONAL RESOURCES INFORMATION CENTER (ERIC) This document has been reproducedas received from the person or organization originating it. 0 Minor changes have been made to improve reproduction quality. Points of view or opinions stated in this document do not necessarily represent official OERI position or policy. PERMISSION TO REPRODUCE AND DISSEMINATE THIS MATERIAL HAS BEEN GRANTED BY 77-1/16A/4-5 A. 0 kj4-ict_ TO THE EDUCATIONAL RESOURCES INFORMATION CENTER (ERIC) Examining the Invariance of Rater and Project Calibrations Using a Multi-Facet Rasch Model Thomas R. O'Neill and Mary E. Lunz American Society of Clinical Pathologists Paper pmsented at the Annual Meeting of the American Educational Research Association, New York, NY. April 1996. 2 BEST COPY AVAILABLE Version: March 21, 1996 Examining the Invariance of Rater and Project Calibrations Using a Multi-Faceted Rasch Model