Jointly Learning to Embed and Predict with Multiple Languages
Daniel C. Ferreira, André F. T. Martins, Mariana S. C. Almeida · 2016
We propose a joint formulation for learning task-specific cross-lingual word embeddings, along with classifiers for that task.Unlike prior work, which first learns the embeddings from parallel data and then plugs them in a supervised learning problem, our approach is oneshot: a single optimization problem combines a co-regularizer for the multilingual embeddings with a task-specific loss.We present theoretical results showing the limitation of Euclidean co-regularizers to increase the embedding dimension, a limitation which does not exist for other co-regularizers (such as the ℓ 1distance).Despite its simplicity, our method achieves state-of-the-art accuracies on the RCV1/RCV2 dataset when transferring from English to German, with training times below 1 minute.On the TED Corpus, we obtain the highest reported scores on 10 out of 11 languages.