Insights From The Broadcast News Benchmark Tests
Walter Liggett, William M. Fisher · 1998
The broadcast news benchmark tests have potential as a source of ideas for improving continuous speech recognition systems. This paper presents a data analysis method for uncovering such ideas and applies the method to the 1996 and 1997 DARPA CSR Hub-4 results. The method is based on a latent variables model instead of a more familiar regression model. The method identifies certain portions of the test material that result in wide performance differences among systems. Such portions, because some systems could handle them and others could not, are worth thinking about in terms of what system features lead to the performance differences. Identification of specific system differences that are responsible for performance differences may lead to system improvements. 1. INTRODUCTION Benchmark tests of continuous speech recognition systems usually entail having each system transcribe the same selection of speech. In the case considered here, the selection is from broadcast news. The system...