Jeff Da at COIN - Shared Task
Jeff Da · 2019
We introduce a simple yet effective method of integrating contextual embeddings with commonsense graph embeddings, dubbed BERT Infused Graphs: Matching Over Other em-beDdings.First, we introduce a preprocessing method to improve the speed of querying knowledge bases.Then, we develop a method of creating knowledge embeddings from each knowledge base.We introduce a method of aligning tokens between two misaligned tokenization methods.Finally, we contribute a method of contextualizing BERT after combining with knowledge base embeddings.We also show BERTs tendency to correct lower accuracy question types.Our model achieves a higher accuracy than BERT, and we score fifth on the official leaderboard of the shared task and score the highest without any additional language model pretraining.