Variational Approach to Factor Analysis and Related Models
Frederik Brink Nielsen · 2004
This thesis focuses on factor analysis, extended factor analysis and parallel factor analysis. For these models Expectation maximization and variational Bayes algorithms are derived. The variational algorithms are implemented with hierarchical automatic relevance determination priors in order to automatically infer the model order. The algorithms are tested on synthetic data and used for analyzing amino acids and fMRI data. In a model selection task the variational lower bound on the marginal likelihood inferred, in a reliable manner, the correct model type. On the other hand, it was found that the lower bound under estimated the model order compared to the Bayesian information criterion. The eUect of hyperparameter optimiza- tion is discussed in relation to the generalization error. Furthermore, it is found that the variational algorithms have properties similar to independent component analysis. A model called probabilistic partial least squares is proposed and used in a NPAIRS analysis of the fMRI data. The NPAIRS analysis and the parallel factor model were found to give very similar results.