The Female Effect
Maria Francesca Roig-Maimó, Ramon Mas-Sansó · 2019
A key aspect when designing experiments is the subjects' selection. Many experiments fail to select a gender-balanced set of subjects and there is often a lower proportion of females. In fact, a significant ratio of papers that include user studies don't even mention the gender distribution. Probably, this is due to the low ratio of females in engineering student's population that normally is used in HCI user studies. In this paper we show that a biased selection of subjects can lead to incorrect results when we rely on users to validate a system addressed to general public. We designed two experiments to obtain a benchmark value of throughput of a head-tracker for mobile devices and we selected a gender-balanced population. To assess the validity of our claim we choose gender as the primary independent variable of a between-subjects study design. We found that gender has a direct impact on throughput; therefore, a gender-unbalanced set of subjects is a threat to the external validity of the results obtained.