Learning-based mechanism design for microtask crowdsourcing
Zehong Hu · 2018
Microtask crowdsourcing, as an efficient and economical method for a requester to outsource tasks to online workers, is becoming increasingly popular in many domains, especially collecting labels for large-scale datasets.In microtask crowdsourcing, a requester usually needs to accomplish three steps: firstly, recruit as many as possible workers from the market; then, assign tasks to the workers based on their performance; lastly, reward good workers and meanwhile punish bad workers.For these three steps, various mechanisms have been proposed.Under certain assumptions about workers' responses to the rewards, these mechanisms can theoretically ensure workers to follow the strategies desired by the requester and thus maximize the revenue of the requester.However, these assumptions may be violated in practice, which causes the failure of these theoretically elegant mechanisms.Thereby, recent studies move their focus to the learning-based mechanisms which learn workers' models in an online fashion rather than simply assuming one.In this thesis, we propose three novel learning-based mechanisms, each for one step, to push forward the studies in this direction.More specifically, when recruiting workers from the microtask crowdsourcing market, an important factor that can be controlled by the requester is the base reward guaranteed for each task, termed the price of tasks.Deciding a proper price is very challenging because overpricing causes inefficient use of the budget, whereas underpricing may lead to an insufficient number of participating workers.To solve this problem, researchers propose posted-price mechanisms which use the classic multiarmed bandit algorithms to learn the worker model online and accordingly adjust the price to be optimal.In this thesis, we propose a novel posted-price mechanism which not only outperforms existing mechanisms on improving the utilities of the requester but also avoids their need for a finite price range as the prior knowledge.The advantages are achieved by designing an optimal multi-armed bandit algorithm to exploit the unique features of microtask crowdsourcing.We theoretically show the optimality of our algorithm and prove that the performance upper bound can be achieved without the need for a prior price range.We also conduct extensive experiments using real price data to verify the advantages and practicability of our mechanism.First and foremost, I would like to take this chance to express my sincere gratitude to my supervisor, Dr. Jie Zhang.He always encourages me by his passion and enthusiasm on research.He also inspire me by his incredibly sharp perceptions on various research problems.Without his support and insightful suggestions, it would be impossible for me to come to the final stage.In addition to research, his way of treating work, friends and students will always be a good guidance for me.