A Seed-Based Method for Generating Chinese Confusion Sets
Liangliang Liu, Cungen Cao · ACM Transactions on Asian and Low-Resource Language Information Processing · 2016
In natural language, people often misuse a word (called a “confused word”) in place of other words (called “confusing words”). In misspelling corrections, many approaches to finding and correcting misspelling errors are based on a simple notion called a “confusion set.” The confusion set of a confused word consists of confusing words. In this article, we propose a new method of building Chinese character confusion sets. Our method is composed of two major phases. In the first phase, we build a list of seed confusion sets for each Chinese character, which is based on measuring similarity in character pinyin or similarity in character shape. In this phase, all confusion sets are constructed manually, and the confusion sets are organized into a graph, called a “seed confusion graph” (SCG), in which vertices denote characters and edges are pairs of characters in the form (confused character, confusing character). In the second phase, we extend the SCG by acquiring more pairs of (confused character, confusing character) from a large Chinese corpus. For this, we use several word patterns (or patterns) to generate new confusion pairs and then verify the pairs before adding them into a SCG. Comprehensive experiments show that our method of extending confusion sets is effective. Also, we shall use the confusion sets in Chinese misspelling corrections to show the utility of our method.