Compositional Distributional Semantics with Long Short Term Memory
Phong Ba Le, Willem H. Zuidema · 2015
We are proposing an extension of the recursive neural network that makes use of a variant of the long short-term memory architecture.The extension allows information low in parse trees to be stored in a memory register (the 'memory cell') and used much later higher up in the parse tree.This provides a solution to the vanishing gradient problem and allows the network to capture long range dependencies.Experimental results show that our composition outperformed the traditional neural-network composition on the Stanford Sentiment Treebank.