Acoustic events semantic detection, classification, and annotation for persistent surveillance applications
Amjad H Alkilani, Amir Shirkhodaie · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · 2014
Understanding of group activity based on analysis of spatiotemporally correlated acoustic sound events has received a minimum attention in the literature and hence is not well understood. Identification of group sub-activities such as: Human-Vehicle Interactions (HVI), Human-Object Interactions (HOI), and Human-Human Interactions (HHI) can significantly improve Situational Awareness (SA) in Persistent Surveillance Systems (PSS). In this paper, salient sound events associated with group activities are preliminary identified and applied for training a Gaussian Mixture Model (GMM) whose features are employed as feature vectors for training of algorithms for acoustic sound recognition. In this paper, discrimination of salient sounds associated with the HVI, HHI, and HOI events is achieved via a Correlation Based Template Matching (CMTM) classifier. To interlinked salient events representing an ontology-based hypothesis, a Hidden Markov Model (HMM) is employed to recognize spatiotemporally correlated events. Once such a connection is established, then, the system generates an annotation of each perceived sound event. This paper discusses the technical aspects of this approach and presents the experimental results for several outdoor group activities monitored by an array of acoustic sensors.