Rethinking Mastery Learning Reconsidered
Thomas R. Guskey · Review of Educational Research · 1987
Best-evidence was proposed by Slavin (1986a) as an alternative to meta-analysis and traditional review techniques. The procedure is described as an extension of the principle in law, where it is a well-established maxim that not all evidence carries equal validity in regard to a particular case. But the way this principle is applied in law is quite different from its application in a best-evidence synthesis. In law, cases are argued and debated. In criminal cases, for example, both the prosecution and the defense present what each believes to be the most valid, evidence. Seldom is the evidence presented by each the same, nor are the conclusions each has drawn from that evidence. It is left to a judge, panel of judges, or jury to weigh all of the evidence and to decide, in as objective and dispassionate a manner as is possible, what evidence is truly best and what verdict or conclusion is to be drawn from that evidence. In a best-evidence synthesis as proposed by Slavin, however, the reviewer serves as judge and jury in determining what evidence is and, hence, what conclusions are to be drawn. Although the reviewer is required to provide detailed specification of the criteria used in selecting the evidence, one can easily imagine two (or three, or a dozen) equally competent best-evidence reviewers' reaching very different conclusions with regard to exactly the same topic simply because they employed different criteria in selecting the evidence. Slavin's (1987) review of mastery learning illustrates clearly the problems inherent in this approach to research synthesis and how the technique can be misused. Suppose, for example, that another best-evidence reviewer of mastery learning research were to choose 3 weeks, rather than 4 (Slavin, 1987), as the minimum duration period for studies to be included. Although neither can be considered an extended time period, in terms of practicality the difference is insignificant. The choice of a 4-week minimum appears arbitrary at best. But given that other more inclusive reviews (Block & Burns, 1976; Guskey & Pigott, in review) have identified a fairly large number of studies of 3-week duration that yielded very positive results, it represents a likely source of reviewer bias at worst. This is not to imply that questions of methodological quality and relevance to the issue should be ignored in research syntheses. These are clearly important considerations in evaluating the validity of findings in a collection of investigations. It is to imply, however, that the term in best-evidence synthesis is highly subjective, potentially biased, and very misleading, especially to those who do not carefully scrutinize the selection criteria employed by the reviewer.