Multimodal blind source separation with a circular microphone array and robust beamforming
Syed Mohsen Naqvi, Muhammad Salman Khan, Qingju Liu, Wenwu Wang, Jonathon A. Chambers · Surrey Research Insight Open Access (The University of Surrey) · 2011
A novel multimodal (audio-visual) approach to the problem of blind source separation (BSS) is evaluated in room en-vironments. The main challenges of BSS in realistic envi-ronments are: 1) sources are moving in complex motions and 2) the room impulse responses are long. For moving sources the unmixing filters to separate the audio signals are difficult to calculate from only statistical information avail-able from a limited number of audio samples. For physically stationary sources measured in rooms with long impulse re-sponses, the performance of audio only BSS methods is lim-ited. Therefore, visual modality is utilized to facilitate the separation. The movement of the sources is detected with a 3-D tracker based on a Markov Chain Monte Carlo par-ticle filter (MCMC-PF), and the direction of arrival infor-mation of the sources to the microphone array is estimated. A robust least squares frequency invariant data independent (RLSFIDI) beamformer is implemented to perform real time speech enhancement. The uncertainties in source localiza-tion and direction of arrival information are also controlled by using a convex optimization approach in the beamformer design. A 16 element circular array configuration is used. Simulation studies based on objective and subjective mea-sures confirm the advantage of beamforming based process-ing over conventional BSS methods. 1.