End-to-end NLP Pipelines in Rust
Guillaume Becquin · 2020
The recent progress in natural language processing research has been supported by the development of a rich open source ecosystem in Python.Libraries allowing NLP practitioners but also non-specialists to leverage stateof-the-art models have been instrumental in the democratization of this technology.The maturity of the open-source NLP ecosystem however varies between languages.This work proposes a new open-source library aimed at bringing state-of-the-art NLP to Rust.Rust is a systems programming language for which the foundations required to build machine learning applications are available but still lacks readyto-use, end-to-end NLP libraries.The proposed library, rust-bert, implements modern language models and ready-to-use pipelines (for example translation or summarization).This allows further development by the Rust community from both NLP experts and nonspecialists.It is hoped that this library will accelerate the development of the NLP ecosystem in Rust.The library is under active development and available at https://github. com/guillaume-be/rust-bert.