Group rotation management in real-time crowdsourcing
Katsumi Kumai, Jianwei Zhang, Yuhki Shiraishi, Daisuke Wakatsuki, Hiroyuki Kitagawa, Atsuyuki Morishima · 2017
A common workflow to perform a continuous human task stream is to divide workers into groups, have one group perform the newly-arrived task, and rotate the groups. Usually, more than one worker belongs to each group for improving the quality of task results. We call this type of workflow the group rotation. This paper addresses the problem of how to manage Group Rotation Type Crowdsourcing, the group rotation in a crowdsourcing setting. In the group-rotation type crowdsourcing, we must change the group structure dynamically because workers come in and leave frequently. However, changing the group structure will give workers psychological stress, such as surprise, confusion or irritation. This paper explores a design space for group restructuring algorithms in the group rotation type crowdsourcing and compares implemented strategies in terms of the evaluation results on psychological stress with real-world crowd workers.