Exploiting Domain Knowledge via Grouped Weight Sharing with Application to Text Categorization
Ye Zhang, Matthew Lease, Byron Wallace · 2017
A fundamental advantage of neural models for NLP is their ability to learn representations from scratch.However, in practice this often means ignoring existing external linguistic resources, e.g., Word-Net or domain specific ontologies such as the Unified Medical Language System (UMLS).We propose a general, novel method for exploiting such resources via weight sharing.Prior work on weight sharing in neural networks has considered it largely as a means of model compression.In contrast, we treat weight sharing as a flexible mechanism for incorporating prior knowledge into neural models.We show that this approach consistently yields improved performance on classification tasks compared to baseline strategies that do not exploit weight sharing.