On the concept of correct hits in spoken term detection
Gábor Gosztolya · SZTE Publicatio Repozitórium (University of Szeged) · 2014
In most Information Retrieval (IR) tasks the aim is to find human-comprehensible items of information in large archives.One such task is the spoken term detection (STD) one, where we look for userentered keywords in a large audio database.To evaluate the performance of a spoken term detection system we have to know the real occurrences of the keywords entered.Although there are standard automatic ways to obtain these locations, it is not obvious how these match user expectations.In our study, we asked a number of subjects to locate these relevant occurrences, and we compared the performance of our spoken term detection system using their responses.In addition, we investigated the nature of their answers, seeking to find a way to determine a commonly accepted list of relevant occurrences.