Proper names and their Spelling Variations in Automatic Speech Recognition output
Hema Raghavan, James Allan · 2004
Names, particularly foreign names often have ambiguous spellings in English. In Automatic Speech Recognized (ASR) documents this ambiguity is more pronounced because a speech recognizer usually chooses similar sounding names or words for those it does not find in its lexicon. The result then is a large number of different spellings for the same name, even within one document. In this paper we propose several methods of normalizing names in an ASR corpus-i.e., grouping together names such as Arafat, Araafat etc which are spelling variations of the same name. Our methods range from a simple String Edit Distance model to more complex generative models that model the corruption in the spelling of names as the effect of a noisy channel. We evaluate our methods using the Paice methodology which was developed for stemming algorithms. We also demonstrate the usefulness of our methods on two tasks- a new task which attempts to find all documents containing all variants of a given name, and the traditional spoken document retrieval task. We get significant improvements on the first task. We also illustrate with several examples the nature of ASR errors and give reasons why ASR errors have not surfaced as a significant problem in the TREC Spoken Document Retrieval task and on the TDT tasks.