Overview of the TREC 2012 Crowdsourcing Track.
Mark D. Smucker, Gabriella Kazai, Matthew Lease · 2012
In 2012, the Crowdsourcing track had two separate tasks: a text relevance assessing task (TRAT) and an image relevance assessing task (IRAT). This track overview describes the track and provides analysis of the track’s results. 1