Advanced Methods of Designing Tests for Pass/Fail Decisions
Nathan Alan Thompson · 2010
Most examinations in the realm of professional regulatory testing are designed with the purpose of pass/fail decisions for each examinee. Historically, these decisions have been made by administering a fixed number of items to each examinee, and assigning a “pass” decision if the examinee observed score is equal to or above a cutscore. However, this method is inefficient, in that it requires a large number of items to be administered before the decision is made. It is for this reason that computerized adaptive testing (CAT) methods were developed. Yet even CAT methods are not optimal for regulatory testing, as they are designed to obtain precise scores, and precise scores are not needed – only a pass/fail decision. A related methodology, computerized classification testing (CCT), specifically designs tests to provide this decision with as few items as possible, but retaining the decision accuracy of both fixed-form and CAT methods. This paper will explain differences between these approaches and provide a comparison of them to demonstrate the efficiency of the CCT approach for pass/fail testing. Simulations will be conducted for tests under each approach, and results compared in terms of decision accuracy (percentage correctly classified) and efficiency (average test length). The applicability of each method in the licensure/certification context will be discussed. Participants will be able to recognize the advantages and disadvantages to help determine if CCT or CAT methods might be useful for their testing program.