Evaluation of ASR Systems using Information Retrieval Measures
Artem Peregoudov · 2005
ABSTRACT Performance evaluation measures play an essential role in the design of automatic speech recognition systems as they are used to predict the performance of the system in a real application, to compare systems and to analyse the errors introduced during the recognition process. Commonly used word error rate (WER) has proven to be a good measure for most ‘old-school’ applications, such as dictation systems, where the whole content is of interest and every word has the same importance. However, with advances made in the recent past, more systems are currently emerging, aiming to resolve a new range of tasks. Multimodal systems, spoken document retrieval and call routing are examples of applications where the task involves categorisation and indexing of the audio content and where not all of the words have the same importance anymore. In this work we investigate on the evaluation of several ASR systems on the Ressource Management task using information retrieval measures.