A method of incorporating bigram constraints into an LR table and its effectiveness in natural language processing
Hiroki IMAI, Hozumi Tanaka · 1998
In this paper, we propose a method for constructing bigram LR tables by way of incorporating bigram constraints into an LR table.Using a bigram LR table, it is possible for a GLR parser to make use of both big'ram and CFG constraints in natural language processing.Applying bigram LR tables to our GLR method has the following advantages:(1) Language models utilizing bigzam LR tables have lower perplexity than simple bigram language models, since local constraints (higram) and global constraints (CFG) are combined in a single bigram LR table.(2) Bigram constraints are easily acquired from a given corpus.Therefore data sparseness is not likely to arise.(3) Separation of local and global constraints keeps down the number of CFG rules.The first advantage leads to a reduction in complexity, and as the result, better performance in GLR parsing.Our experiments demonstrate the effectiveness of our method.