Multi-Mode Transformer Transducer with Stochastic Future Context
Kwangyoun Kim, Felix Wu, Prashant Sridhar, Kyu Jeong Han, Shinji Watanabe · 2021
Automatic speech recognition (ASR) models make fewer errors when more surrounding speech information is presented as context.Unfortunately, acquiring a larger future context leads to higher latency.There exists an inevitable trade-off between speed and accuracy.Naïvely, to fit different latency requirements, people have to store multiple models and pick the best one under the constraints.Instead, a more desirable approach is to have a single model that can dynamically adjust its latency based on different constraints, which we refer to as Multimode ASR.A Multi-mode ASR model can fulfill various latency requirements during inference -when a larger latency becomes acceptable, the model can process longer future context to achieve higher accuracy and when a latency budget is not flexible, the model can be less dependent on future context but still achieve reliable accuracy.In pursuit of Multi-mode ASR, we propose Stochastic Future Context, a simple training procedure that samples one streaming configuration in each iteration.Through extensive experiments on AISHELL-1 and Lib-riSpeech datasets, we show that a Multi-mode ASR model rivals, if not surpasses, a set of competitive streaming baselines trained with different latency budgets.