SCALE: A Scalable Language Engineering Toolkit
Joris Pelemans, Lyan Verwimp, Kris Demuynck, Hugo Van hamme, Patrick Wambacq · 2016
In this paper we present SCALE, a new Python toolkit that contains two extensions to n-gram language models.The first extension is a novel technique to model compound words called Semantic Head Mapping (SHM).The second extension, Bag-of-Words Language Modeling (BagLM), bundles popular models such as Latent Semantic Analysis and Continuous Skip-grams.Both extensions scale to large data and allow the integration into first-pass ASR decoding.The toolkit is open source, includes working examples and can be found on http://github.com/jorispelemans/scale.