Deciding to Stop Early or Continue the Experiment After Checking p -Values at Interim Points: Introducing Group Sequential Designs to UI-Based Comparative Studies
Shota Yamanaka · International Journal of Human-Computer Interaction · 2024
Null hypothesis significance testing (NHST) is widely used in the field of human-computer interaction (HCI), and caution has been advised against deciding whether to stop or continue an experiment based on checking p-values. However, in clinical trials, NHST with interim analysis is commonly employed by adjusting the α levels to control Type I error rates. This approach, known as a group sequential design, can also be beneficial in the HCI field. We analyzed existing datasets of target-pointing tasks using group sequential designs and experimentally demonstrated that, depending on the α-level correction method, we could reduce the number of participants by 40.4–56.5%. Therefore, employing group sequential designs enables significant time and cost savings for both participants and researchers in the HCI field.