Detecting Items That Have Been Memorized in the Computerized Adaptive Testing Environment.
Lori McLeod, Deborah L. Schnipke · 1999
Because scores on high-stakes tests influence many decisions, tests need to be secure. Decisions based on scores affected by preknowledge of items are unacceptable. New methods are needed to detect the new cheating strategies used for computer-administered tests because item pools are typically used over time, providing the potential opportunity for test takers to share items with future test takers. Because of the serious ramifications of accusing someone as being a user of item preknowledge (or cheater), it may be more useful for operational computer-administered test developers to focus on item security rather than the behavior of individual test takers. This research explores the development and use of a fit index to detect items that have been memorized so that these items may be removed from the item pool, while leaving secure items in the pool. The results from this initial simulation for the developed Bayesian posterior log odds ratio index are promising. It is hoped that this work and future work will enable testing programs to determine more effectively how long to leave an item pool (or specific items) in the field. (Contains four tables, five figures, and five references.) (Author/SLD) ******************************************************************************** Reproductions supplied by EDRS are the best that can be made from the original document. ******************************************************************************** Detecting Items That Have Been Memorized In The Computerized Adaptive Testing Environment Lori D. McLeod Deborah L. Schnipke Law School Admission Council PERMISSION TO REPRODUCE AND DISSEMINATE THIS MATERIAL HAS BEEN G9ANTED BY TO THE EDUCATIONAL RESOURCES INFORMATION CENTER (ERIC) U.S. DEPARTMENT OF EDUCATION Office of Educational Research and improvement EDUCATIONAL RESOURCES INFORMATION CENTER (ERIC) erff.Trs document has been reproduced as received from the person or organization originating it. O Minor changes have been made to improve reproduction quality. Points of view or opinions stated in this document do not necessarily represent official OERI position or policy. Paper presented at the Annual Meeting of National Council on Measurement in Education, Montreal, CA, April 19-23, 1999. 2 BEST COPY AVAILABLE Detecting Memorized Items 2