Optional Stopping and the Interpretation of The Bayes Factor
Jeffrey N. Rouder · 2019
In their paper Why optional stopping is a problem for Bayesians, @deHeide:Grunwald:2020 critique the claim that those using Bayes factor can sample until there is sufficient evidence to support one or another model. de Heide and Grünwald's main message is that unless you believe your priors to govern the data-generating mechanism, optional stopping may distort inference. We show here that the distortions are not in inference but in interpretation. Their claim is about what happens when the analyst's models do not match the data generating models, that is, the Bayes factor for models not specified. Given that we never know the underlying data generation mechanism we ask researcher to interpret Bayes factors for what they are---the relative strength of evidence from data for two hypotheticals. We discuss how researchers should assess the robustness of their results by considering reasonable variation in model specification conditional on observed data rather than on hypothetical truths.