A Deep Network with Visual Text Composition Behavior
Hongyu Guo · 2017
While natural languages are compositional, how state-of-the-art neural models achieve compositionality is still unclear.We propose a deep network, which not only achieves competitive accuracy for text classification, but also exhibits compositional behavior.That is, while creating hierarchical representations of a piece of text, such as a sentence, the lower layers of the network distribute their layer-specific attention weights to individual words.In contrast, the higher layers compose meaningful phrases and clauses, whose lengths increase as the networks get deeper until fully composing the sentence.