Supplemental Material for Speakers and Listeners Exploit Word Order for Communicative Efficiency: A Cross-Linguistic Investigation
Journal of Experimental Psychology General · 2020
Supplemental materials 1. Post-hoc power analysesWe determined the power of our studies by bootstrapping our experimental data for each key prediction.All code for these post-hoc power analyses is available at https://osf.io/9hw68/.All our key predictions (see below) relied on differences between conditions.We analyzed these differences by computing bootstrapped 95% confidence intervals over the difference in effect sizes and seeing if the interval crossed 0 (suggesting there may be no difference in effect sizes) or not.The bootstrap process followed two stages.First, we bootstrapped the data to obtain a replicate of the two target experimental conditions (depending on prediction; see below).Next, for each experimental replicate, we conducted the analysis from the manuscript: bootstrapping the results to obtain a confidence interval over the difference in effect sizes.We ran 5000 experimental replicates and within each replicate we bootstrapped the difference in effect sizes with 1000 replicates.Experiment 1 Prediction 1. Percentage of redundant color words should be different in the four-item displays across languages.In 94.9% of the bootstrapped replicates, the corresponding confidence interval did not cross 0, suggesting that the two effects were reliably different and that our experiment's power is 0.949.Prediction 2. Percentage of redundant color words in English should be higher in the 16item displays relative to the four-item displays.In 92.0% of the bootstrapped replicates, the corresponding confidence interval did not cross 0, suggesting that the two effects were reliably different and that our experiment's power is 0.92.Prediction 3. Percentage of redundant color words in Spanish should be higher in the 16item displays relative to the four-item displays.In 100% of the bootstrapped replicates, the corresponding confidence interval did not cross 0, suggesting that the two effects were reliably different.Naturally, this does not imply that our power is 1, as it is likely that at least one replicate would eventually not produce a difference.Note, however, that the difference across conditions was striking in Spanish (see Figure 1S below), with almost no Spanish speaker producing color words in the four-item display, and the majority of them producing color words more than half of the time in the 16-item display.Given that we found no replicates that went against our predictions in the 5000 bootstrap samples, this suggests that the probability of a replicate not showing our predicted effect is lower than 1/500 = 1e-04.