On the design of robust criteria and algorithms for blind source separation
S.C. Douglas, Jih-Cheng Chao · 2007
Blind Source Separation (BSS) is a class of signal processing methods that recover a set of source signals from their linear mixtures with no or little prior knowledge of the source signals or mixing conditions. Independent Component Analysis (ICA) is a procedure for recovering a set of independent features from multichannel data that is often useful for BSS. In BSS and ICA, it is desirable to select a separation criterion that results in a simple algorithm and achieves accurate and robust source estimates. Many procedures for BSS and ICA have been developed; two of the most-popular methods are the natural gradient algorithm and the FastICA algorithm of Hyvarinen and Oja. Both of these procedures rely on output nonlinearities for each extracted source to obtain separation. These nonlinearities are used to compute the updated demixing coefficients in the separation algorithm. The design of these nonlinearities is largely dependent on the distributions of the original source signals, and the design is often performed in an ad hoc manner without careful regard for the algorithm's behavior in many treatments of the problem. An important design consideration in these algorithms is the choice of nonlinearity used to obtain each estimated output. In this dissertation, we review existing work in the area of BSS and ICA and present novel research results in the use of Huber's M-estimator cost function and piecewise linear function as contrast functions within the natural gradient and FastICA algorithm for separating various types of non-Gaussian sources. The algorithms obtained from these cost functions are particularly simple to implement, as they involve only multiplies, adds, and threshold operations. We establish key properties regarding the local stabilities of the algorithms for general non-Gaussian source distributions, and their separating capabilities are shown through analysis to be largely insensitive to the cost function's threshold parameter. In addition, we show that the Huber M-estimator cost function is able to separate large-scale and ill-conditioned signal mixtures with reduced data set requirements effectively. These key features are used for the blind source separation portion of the first Machine Learning for Signal Processing Workshop Data Analysis Competition and resulted in a winning algorithm for the competition. More significantly, the frequency domain FastICA algorithm with the Huber M-estimator cost is able to separate real-word speech mixtures and outperforms other algorithms based on other contrast costs. Using the Huber M-Estimator cost with the natural gradient algorithm requires careful design of the threshold parameter. We show how the cost can be simply modified to produce a new cost involving piecewise linear functions that has improved stability monitoring properties, guaranteeing local stability with any source distribution through a nonlinearity sign change. Moreover, we show that the deviation of the output nonlinearity from a linear function for the FastICA algorithm can be extremely small, suggesting that simple output nonlinearities already in use, such as mu-law companding, is sufficiently nonlinear to allow separation with such algorithms.