Statistical physics of independent component analysis
R. Urbanczik · 2003
PACS. 89.75.Fb – Structures and organization in complex systems. PACS. 84.35.+i – Neural networks. PACS. 64.60.Cn – Order disorder transitions; statistical mechanics of model systems. Abstract. – Statistical physics is used to investigate independent component analysis with polynomial contrast functions. While the replica method fails, an adapted cavity approach yields valid results. The learning curves, obtained in a suitable thermodynamic limit, display a first order phase transition from poor to perfect generalization. During the last decade, independent component analysis (ICA) has emerged as one of the most powerful unsupervised learning procedure for many signal processing tasks [1,2]. It assumes that the observed, often high dimensional signal, is a linear mixture of independent source signals and aims to recover these sources just from observing the mixed up signal. Hence, ICA is sometimes also called blind signal deconvolution. An illustrative scenario is the cocktail party problem where, to understand any single speaker, we first need to identify her voice amidst the jumble of sounds reaching our ears. The basic finding in ICA is that the distribution of the observed signal will be similar to