Worker types and personality traits in crowdsourcing relevance labels
Gabriella Kazai, Jaap Kamps, Nataša Milić-Frayling · 2011
Crowdsourcing platforms offer unprecedented opportunities for creating evaluation benchmarks, but suffer from varied output quality from crowd workers who possess different levels of competence and aspiration. This raises new challenges for quality control and requires an in-depth understanding of how workers' characteristics relate to the quality of their work.