Is Knowledge-Free Induction of Multiword Unit Dictionary Headwords a Solved Problem?
Patrick Schone, Daniel S. Jurafsky · 2001
We seek a knowledge-free method for inducing multiword units from text corpora for use as machine-readable dictionary headwords. We provide two major evaluations of nine existing collocation-finders and illustrate the continuing need for improvement. We use Latent Semantic Analysis to make modest gains in performance, but we show the significant challenges encountered in trying this approach.