Application Of A Distance-weighted Statistical Language Model
Jin Ling · Zhongwen xinxi xuebao · 2001
Proposed in this paper is a novel language model based on the traditional N gram model,where the inter word distance information is integrated,and therefore the model is referred to as the distance weighted statistical language model.In this model,the relationship between disconnected words is taken into consideration.Based on the principle that closer words(in distance)have a closer relation.A distance weighted function has been used to integrate the information so as to improve the model's prediction ability.Compared with the original n gram model,the experiments results show that the proposed language model will reduce the Chinese whole sentence IME system's word error rate.