Fast Word Lattice Generation in Morphological Analysis
Nobuhiro Kaji, Masaru Kitsuregawa · Transactions of the Japanese Society for Artificial Intelligence · 2014
This paper proposes a fast word lattice generation algorithm for Japanese morphological analysis. We conducted experiments on three Japanese data sets to demonstrate that the previously proposed pruning-based algorithm is in fact not efficient enough, and that the pipeline algorithm, which is introduced in this paper, achieves considerable speed-up without loss of accuracy. Moreover, the compactness of the lattice generated by the pipeline algorithm was investigated from both theoretical and empirical perspectives.