Thematic fit evaluation: an aspect of selectional preferences
Asad Sayeed, Clayton Greenberg, Vera Demberg · 2016
In this paper, we discuss the human thematic fit judgement correlation task in the context of real-valued vector space word representations.Thematic fit is the extent to which an argument fulfils the selectional preference of a verb given a role: for example, how well "cake" fulfils the patient role of "cut".In recent work, systems have been evaluated on this task by finding the correlations of their output judgements with human-collected judgement data.This task is a representationindependent way of evaluating models that can be applied whenever a system score can be generated, and it is applicable wherever predicate-argument relations are significant to performance in end-user tasks.Significant progress has been made on this cognitive modeling task, leaving considerable space for future, more comprehensive types of evaluation.