Measuring semantic content in distributional vectors
Aurélie Herbelot, Mohan Ganesalingam · 2013
Some words are more contentful than others: for instance, make is intuitively more general than produce and fifteen is more ‘precise ’ than a group. In this paper, we propose to measure the ‘semantic content’ of lexical items, as modelled by distributional representations. We investigate the hypothesis that semantic content can be computed using the Kullback-Leibler (KL) divergence, an informationtheoretic measure of the relative entropy of two distributions. In a task focusing on retrieving the correct ordering of hyponym-hypernym pairs, the KL divergence achieves close to 80 % precision but does not outperform a simpler (linguistically unmotivated) frequency measure. We suggest that this result illustrates the rather ‘intensional ’ aspect of distributions. 1