Segmentation, Indexing, and Retrieval for Environmental and Natural Sounds
Gordon Wichern, Jiachen Xue, Harvey Thornburg, Brandon Mechtley, Andreas Spanias · IEEE Transactions on Audio Speech and Language Processing · 2010
We propose a method for characterizing sound activity in fixed spaces through segmentation, indexing, and retrieval of continuous audio recordings. Regardingsegmentation, we present a dynamic Bayesian network (DBN) that jointly infers onsets and end times of the most prominent sound events in the space, along with an extension of the algorithm for covering large spaces with distributed microphone arrays. Each segmented sound event isindexedwith a hidden Markov model (HMM) that models the distribution of example-based queries that a user would employ toretrievethe event (or similar events). In order to increase the efficiency of the retrieval search, we recursively apply a modified spectral clustering algorithm to group similar sound events based on the distance between their corresponding HMMs. We then conduct a formal user study to obtain the relevancy decisions necessary for evaluation of our retrieval algorithm on both automatically and manually segmented sound clips. Furthermore, our segmentation and retrieval algorithms are shown to be effective in both quiet indoor and noisy outdoor recording conditions.