An efficient algorithm for building a distributional thesaurus (and other Sketch Engine developments)
Pavel Rychlý, Adam Kilgarriff · 2007
Gorman and Curran (2006) argue that thesaurus generation for billion+-word corpora is problematic as the full computation takes many days. We present an algorithm with which the computation takes under two hours. We have created, and made publicly available, thesauruses based on large corpora for (at time of writing) seven major world languages. The development is implemented in the Sketch Engine (Kilgarriff et al., 2004).