User - oriented Chinese language model and its machine learning
Xiaolong Wang · Ha'erbin gongye daxue xuebao · 2004
In order to improve the adaptability of the language model, the user - oriented language model is proposed consisting of the general - purpose language model ( with its original parameters kept unchanged) obtained through large - scale training on balanced corpus and the user model ( with its parameters dynamically updated using the first in and first out technique)obtained through on-line learning. In the data storage structure , a multi - level index structure is used in the general - purpose model to solve the data sparseness problem, and the user model is represented by linear structures, and searched by the halving method. A machine method suitable for Chinese N-gram model is proposed following the principle of correcting as much language model transfer errors as possible and avoiding language model imbalance. Experimental results indicate that this machine method has the strengthening characteristics, and provides together with the progressive learning mode a more flexible choice for the application system.