Metrics of games-with-a-purpose for NLP applications
Jon Chamberlain, Richard A. Bartle, Udo Kruschwitz, Chris Madge, Massimo Poesio · Queen Mary Research Online (Queen Mary University of London) · 2017
A Game-With-A-Purpose (GWAP) ap- proach is a promising way to collect data on a large scale but its adoption for NLP annotation projects has been slow. In this paper, we propose evaluation metrics adapted from free-to-play (F2P) games to help understand what makes games suc- cessful for NLP applications.