Social Cues, Social Biases: Stereotypes in Annotations on People Images
Jahna Otterbacher · AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) · 2018
Human computation is often subject to systematic biases. We consider the case of linguistic biases and their consequences for the words that crowd workers use to describe people images in an annotation task. Social psychologists explain that when describing others, the subconscious perpetuation of stereotypes is inevitable, as we describe stereotype-congruent people and/or in-group members more abstractly than others. In an MTurk experiment we show evidence of these biases, which are exacerbated when an image’s “popular tags” are displayed, a common feature used to provide social information to workers. Underscoring recent calls for a deeper examination of the role of training data quality in algorithmic biases, results suggest that it is rather easy to sway human judgment.