A Truly Multilingual First Pass and Monolingual Second Pass Streaming on-Device ASR System
Sepand Mavandadi, Bo Li, Chao Zhang, Brian Farris, Tara N. Sainath, Trevor Strohman · 2022 IEEE Spoken Language Technology Workshop (SLT) · 2023
Automatic speech recognition (ASR) systems need to be accurate, have low latency, and effectively handle language switching in order to be useful for the 60% of the world population that speaks more than one language. Thus, we propose a truly multilingual first-pass and monolingual second-pass streaming on-device ASR system based on the recently developed Cascaded Encoders model. The streaming first-pass recognizes multilingual speech without needing language information, providing real-time transcription, even for code-switching speech. The second-pass uses a language dependent right context encoder to improve the recognition accuracy. On a 9 language Voice Search task, we find that a system combining shared causal encoder with decoders and non-causal encoders replicated per-language reduces word error rate (WER) by 4.4% relative to monolingual baselines. We further show this design to be parameter efficient, outperforming other architectures when matched in the number of parameters.