Replication Value in the Courtroom; a Commentary on Isager, van 't Veer & Lakens
E. Beerdsen · 2024
Isager, van ‘t Veer, and Lakens (2021) propose a sensible and simple approach to help determine which claims to prioritize for replication studies, based primarily on citation counts (as a measure of value) and sample sizes (as a measure of uncertainty). When it comes to using citation counts as a measure of value, the authors acknowledge that clinical, political, and educational impact and other sources of value are unlikely to be captured fully by citation counts (Isager et al., 2021, pp. 10-12). In this Commentary, I focus on the impact that original research can have in a legal context. I argue that citation counts represent a promising start for evaluating the value of a claim, but that it is important to try to capture nonscientific impact as part of assessing value, even if doing so will be more complex than assessing scientific impact. A full accounting for nonscientific impact will likely require significant follow-up work, but I believe that a beginning can be made within Isager et al.’s proposed framework.