How to Make Decisions About the Effectiveness of Device-Based Training: Elaborations on What Everybody Knows
John A. Boldovici, Eugenia M. Kolasinski · Military Psychology · 1997
Statisticians, biomedical researchers, and behavioral scientists have publicized errors in examinations of the differential effects of 2 or more treatments. Publicity about those errors seems to have been ignored by many applied behavioral researchers, including some responsible for evaluations of device-based training in the U.S. Army. Ignoring the causes and effects of the common evaluation errors, and especially errors associated with hypotheses of equal effectiveness of conventional training and device-based training, leads to logical contradictions, threats to readiness, and a lack of scientifically legitimate ways to examine the effects of OPTEMPO (operating tempo) alterations. Those problems may be avoided by applying a few basics of statistical analysis and inference to the design and interpretation of evaluations of device-based training. The basics include hypothesis tests, power analyses, and confidence intervals; they are elaborated in this article with examples of how to apply each in designing and interpreting evaluations of the Army’s forthcoming Close Combat Tactical Trainer.