Assessing the Comparability of News Texts
Emma Barker, Robert Gaizauskas · 2012
Comparable news texts are frequently proposed as a potential source of alignable sub-sentential fragments for use in statistical machine translation systems.But can we assess just how potentially useful they will be?In this paper we first discuss a scheme for classifying news text pairs according to the degree of relatedness of the events they report and investigate how robust this classification scheme is via a multi-lingual annotation exercise.We then propose an annotation methodology, similar to that used in summarization evaluation, to allow us to identify and quantify shared content at the sub-sentential level in news text pairs and report a preliminary exercise to assess this method.We conclude by discussing how this works fits into a broader programme of assessing the potential utility of comparable news texts for extracting paraphrases/translational equivalents for use in language processing applications.