Learning Hierarchical Structures On-The-Fly with a Recurrent-Recursive Model for Sequences
Athul Paul Jacob, Zhouhan Lin, Alessandro Sordoni, Yoshua Bengio · 2018
We propose a hierarchical model for sequential data that learns a tree on-thefly, i.e. while reading the sequence.In the model, a recurrent network adapts its structure and reuses recurrent weights in a recursive manner.This creates adaptive skip-connections that ease the learning of long-term dependencies.The tree structure can either be inferred without supervision through reinforcement learning, or learned in a supervised manner.We provide preliminary experiments in a novel Math Expression Evaluation (MEE) task, which is explicitly crafted to have a hierarchical tree structure that can be used to study the effectiveness of our model.Additionally, we test our model in a wellknown propositional logic and language modelling tasks.Experimental results show the potential of our approach.