Diversity-aware Evaluation for Paraphrase Patterns
Hideki Shima, Teruko Mitamura · 2011
Common evaluation metrics for paraphrase patterns do not necessarily correlate with extrinsic recognition task performance. We propose a metric which gives weight to lexical variety in paraphrase patterns; our proposed metric has a positive correlation with paraphrase recognition task performance, with a Pearson correlation of 0.5~0.7 (k=10, with “strict ” judgment) in a statistically significant level (p-value<0.01). 1