JointGT: Graph-Text Joint Representation Learning for Text Generation from Knowledge Graphs
Pei Ke, Haozhe Ji, Ran Yu, Xin Guo Cui, Liwei Wang, Linfeng Song, Xiaoyan Zhu, Minlie Huang · 2021
Existing pre-trained models for knowledgegraph-to-text (KG-to-text) generation simply fine-tune text-to-text pre-trained models such as BART or T5 on KG-to-text datasets, which largely ignore the graph structure during encoding and lack elaborate pre-training tasks to explicitly model graph-text alignments.To tackle these problems, we propose a graph-text joint representation learning model called JointGT.During encoding, we devise a structure-aware semantic aggregation module which is plugged into each Transformer layer to preserve the graph structure.Furthermore, we propose three new pre-training tasks to explicitly enhance the graph-text alignment including respective text / graph reconstruction, and graph-text alignment in the embedding space via Optimal Transport.Experiments show that JointGT obtains new stateof-the-art performance on various KG-to-text datasets 1 .