Structure-aware Fine-tuning of Sequence-to-sequence Transformers for Transition-based AMR Parsing
Jiawei Zhou, Tahira Naseem, Ramón Fernández Astudillo, Young‐Suk Lee, Radu Florian, Salim Roukos · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · 2021
Predicting linearized Abstract Meaning Representation (AMR) graphs using pre-trained sequence-to-sequence Transformer models has recently led to large improvements on AMR parsing benchmarks.These parsers are simple and avoid explicit modeling of structure but lack desirable properties such as graph well-formedness guarantees or built-in graph-sentence alignments.In this work we explore the integration of general pre-trained sequence-to-sequence language models and a structure-aware transition-based approach.We depart from a pointer-based transition system and propose a simplified transition set, designed to better exploit pre-trained language models for structured fine-tuning.We also explore modeling the parser state within the pre-trained encoder-decoder architecture and different vocabulary strategies for the same purpose.We provide a detailed comparison with recent progress in AMR parsing and show that the proposed parser retains the desirable properties of previous transition-based approaches, while being simpler and reaching the new parsing state of the art for AMR 2.0, without the need for graph re-categorization.