Techniques for two-stage open vocabulary spoken term detection and verification
Atta Norouzian · eScholarship@McGill (McGill) · 2015
Spoken term detection (STD) is one of many applications that require a capability for searchand retrieval of spoken content from large media repositories. In a typical STD scenario, auser enters a query term consisting of a word or phrase and, in response, the search enginereturns a list of detected occurrences of the query term in the repository. The state-ofthe-art STD systems use an automatic speech recognition (ASR) system for generatinga tokenized representation of the speech and perform search on this representation tofind hypothesized occurrences of the query terms. Varying acoustic conditions, speakerpopulations, and speaking styles, along with specialized task domains, all contribute togenerally poor speech recognition performance in many STD scenarios. Furthermore, thesize of media repositories can be extremely large, in some cases on the order of thousandsof hours of audio material. These would reduce the search accuracy and speed respectivelyin ASR-based STD systems. The objective of this thesis is to address these issues.The work presented in this thesis constitutes four major contributions. The first is thedevelopment of a fast and accurate ASR-based STD approach for large audio repositories.This approach is based on ecient indexing of ASR outputs and a two-stage phonemebasedsearch procedure which facilitates detecting occurrences of all query terms, whetherthey belong to the ASR vocabulary or not. The second contribution is the developmentof a graph-based approach for verifying the occurrence of query terms in the set of candidatespeech intervals derived from an STD system. In this approach, the confidence scoresassociated with the hypothesized query term occurrences, generated by the original STDsystem, are adjusted based on the acoustic similarity of the corresponding acoustic intervalsto each other and to other intervals in the repository. The third contribution of this thesisis the use of a feature representation and modeling formalism, distinct from those used inconventional ASR systems, for generating alternative confidence scores for a given set ofhypothesized query term occurrences. It is shown that the resulting confidence scores arecomplementary to the confidence scores estimated in conventional ASR-based STD systems.The fourth contribution is the development of two manifold-based semi-supervisedapproaches for verifying hypothesized occurrences of query terms. It is demonstrated thatdeploying unlabeled data in addition to labeled data in training term-dependent modelsunder the proposed semi-supervised framework improves the verification accuracy. Moreover,in extremely low-resource scenarios, reasonably good STD performance is achieved by only exploiting the similarity of the hypothesized query term occurrences using a semisupervisedapproach based on graph spectral clustering.