The User Experience of a Crowdsourced Photo Tagging Game: Evaluation with Usability Tests
Lotta Söderholm · Työväentutkimus Vuosikirja · 2013
There are big amounts of photos taken and uploaded to the internet every year, but few of them are labelled in any way that makes searching for them possible. At the same time people spent hours playing casual computer games like Solitaire or Mahjong. Phototag, a memorygame, was designed to take advantage of this by crowdsourcing the tagging of photos. A prototype of the game has been developed and the purpose of this study was to evaluate the entertainment value of the game and the value of the tags produced during game play. The tests were done with two groups of participants. One group played the game with random photos and another group with family members' photos. The tests were conducted by modifying classical usability evaluation methods to fit the evaluation of the user experience of the game. Data was collected using think-aloud protocol, video material, questionnaires and logged data from the game. The tags added during game play were evaluated by the participants and the researcher. The tests showed that the game had some playability problems: efficiency, learnability and memorability problems were found. A lot of unnecessary clicking was required during the game, which made the players easily frustrated. The players also found the writing of tags tiresome and many tried playing the game without adding any tags. Some of the participants expressed that they liked the game and would play it again, whereas others said they would not. The participants that normally like playing more casual games were also more satisfied with the game. Most of the tags produced describes the photos well and were the same tags the original photograph had added to the photos. More than half of the tags could be used as such, but there were also poor tags added such as codewords, acronyms, typos and words that only meant something to the player. This shows that the tags would need to go through quality checks before they can be used.