A Meta-Analysis of Effect Sizes of CHI Typing Experiments
Natalia Obukhova · 2021
While designing an HCI experiment, planning the sample size with a priori power analysis is often skipped due to the lack of reference effect sizes. On the one hand, it can lead to a false-negative result, missing the effect that is present in the population. On the other hand, it poses a risk of spending more resources if the number of participants is too high. In this work, I present the reference for small, medium, and large effect sizes for typing experiments based on a meta-analysis of well-cited papers from CHI conference. This effect size ruler can be used to conduct a priori power analysis or assess the magnitude of the found effect. This work also includes comparisons to other fields and conclude with a discussion of the existing issues with reporting practices and data availability. This paper and all data and materials are freely available at https://osf.io/nqzpr.