ICA through an LS-SVM based Kernel CCA Measure for Independence
Carlos M. Alzate, Johan A. K. Suykens · IEEE International Conference on Neural Networks/IEEE ... International Conference on Neural Networks · 2007
A new measure for independence based on canonical correlation in high dimensional feature spaces is presented. This measure can be used as a contrast function for independent component analysis (ICA). The formulation fits in the least squares support vector machines (LS-SVM) framework as a primal-dual interpretation of kernel canonical correlation analysis (CCA) in the context of constrained optimization problems. Regularization is incorporated naturally in the primal formulation leading to a dual generalized eigenvalue problem. Due to the primal-dual nature of the proposed approach, the measure for independence can be calculated for out-of-sample data points which is important for parameter selection ensuring statistical reliability of the estimated measure. Simulations results with small toy datasets performing model selection on a validation set showed good performance avoiding overfltting. Experiments with image demixing using approximated kernel matrices via incomplete Cholesky decomposition showed good results together with a reduced computational cost.