Learning New Semi-Supervised Deep Auto-encoder Features for Statistical Machine Translation
Shixiang Lü, Zhenbiao Chen, Bo Xu · 2014
In this paper, instead of designing new features based on intuition, linguistic knowledge and domain, we learn some new and effective features using the deep autoencoder (DAE) paradigm for phrase-based translation model.Using the unsupervised pre-trained deep belief net (DBN) to initialize DAE's parameters and using the input original phrase features as a teacher for semi-supervised fine-tuning, we learn new semi-supervised DAE features, which are more effective and stable than the unsupervised DBN features.Moreover, to learn high dimensional feature representation, we introduce a natural horizontal composition of more DAEs for large hidden layers feature learning.On two Chinese-English tasks, our semi-supervised DAE features obtain statistically significant improvements of 1.34/2.45(IWSLT) and 0.82/1.52(NIST) BLEU points over the unsupervised DBN features and the baseline features, respectively.