Alignment-Based Discriminative String Similarity
Shane Bergsma, Grzegorz Kondrak · Meeting of the Association for Computational Linguistics · 2007
A character-based measure of similarity is an important component of many natural language processing systems, including approaches to transliteration, coreference, word alignment, spelling correction, and the identication of cognates in related vocabularies. We propose an alignment-based discriminative framework for string similarity. We gather features from substring pairs consistent with a character-based alignment of the two strings. This approach achieves exceptional performance; on nine separate cognate identication experiments using six language pairs, we more than double the precision of traditional orthographic measures like Longest Common Subsequence Ratio and Dice’s Coefcient. We also show strong improvements over other recent discriminative and heuristic similarity functions.