Benchmark test procedures for automatic speech recognition systems
David S. Pallett · The Journal of the Acoustical Society of America · 1988
Benchmark test procedures have been developed and implemented in the DARPA-sponsored program in automatic recognition of continuous speech. These tests make use of a database of recorded speech that includes material for both speaker-dependent and speaker-independent technologies. In implementing these uniform tests, it is necessary to define a standard lexicon of units to be scored and a standard or reference orthographic representation for each sentence, and to select a scoring procedure and error taxonomy. Considerations taken in identifying the reference lexicon and representations will be outlined. For the DARPA program, a dynamic-programming string alignment procedure was implemented, and errors were defined in terms of substitutions, insertions, and deletions. This scoring software is publicly available. Known limitations of this procedure will be discussed. [Work supported by DARPA.]