Multi-Model Comparison Using the Cross-Fitting Method
Holger Schultheis, Praneeth Naidu · eScholarship (California Digital Library) · 2014
When comparing the ability of computational cognitive models to fit empirical data, the complexity of the compared models needs to be taken into account.A promising method for achieving this is the parametric bootstrap cross-fitting method (PBCM) proposed by Wagenmakers, Ratcliff, Gomez, and Iverson (2004).We contribute to a wider applicability of the PBCM in two ways: First, we compare the performance of the data-informed and the data-uninformed variant of the PBCM.Our simulations suggest that only the data-uninformed variant successfully controls for model complexity in model selection.Second, we propose an extension of the PBCM, called MMP-BCM, that is applicable to, in principle, arbitrarily many competing models.We evaluate the MMPBCM by applying it to the comparison of several sets of competing models.The obtained results suggest that the MMPBCM constitutes a more powerful approach to model comparison than the PBCM.