Multiview acoustic feature learning using articulatory measurements
Sujeeth Bharadwaj, Raman Arora, Karen Livescu, Mark Hasegawa‐Johnson · 2012
We consider the problem of learning a linear transformation of acoustic feature vectors for phonetic classification, in a setting where articulatory measurements are available at training time. We use the acoustic and articulatory data together in a multi-view learning approach, in particular using canonical correlation analysis to learn linear transformations of the acoustic features that are maximally correlated with the articulatory data. We also investigate simple approaches for combining information shared across the acoustic and articulatory views with information that is private to the acoustic view. We apply these methods to phonetic frame classification on data drawn from the University of Wisconsin X-ray Microbeam Database. We find a small but consistent advantage to the multi-view approaches combining shared and private information, compared to the baseline acoustic features or unsupervised dimensionality reduction using principal components analysis. Index Terms — Multi-view learning, canonical correlation analysis, articulatory measurements, dimensionality reduction, acoustic features 1.