An automatic chinese collocation extraction algorithm based on lexical statistics
Ruifeng Xu, Qin Lu, Li Yin · 2004
We present an automatic Chinese collocation extraction system using lexical statistics and syntactical knowledge. This system extracts collocations from manually segmented and tagged Chinese news corpus in three stages. First, the bidirectional bigram statistical measures, including bidirectional strength and spread, and /spl chi//sup 2/ test value, are employed to extract candidate two-word pairs. These candidate word pairs are then used to extract high frequency multiword collocations from their context. In the third stage, precision is further improved by using syntactical knowledge of collocation patterns between content words to eliminate pseudo collocations. In the preliminary experiment on 30 selected headwords, this three-stage system achieves a 73% precision rate, a substantial improvement on the 61% achieved using an algorithm we developed earlier based on an improved version of the Smdja's 53% accurate Xtract system.