MetaTS: Meta Teacher-Student Network for Multilingual Sequence Labeling with Minimal Supervision
Zheng Li, Danqing Zhang, Tianyu Cao, Ying Wei, Yiwei Song, Bing Yin · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · 2021
Sequence labeling aims to predict a finegrained sequence of labels for the text.However, such formulation hinders the effectiveness of supervised methods due to the lack of token-level annotated data.This is exacerbated when we meet a diverse range of languages.In this work, we explore multilingual sequence labeling with minimal supervision using a single unified model for multiple languages.Specifically, we propose a Meta Teacher-Student (MetaTS) Network, a novel meta learning method to alleviate data scarcity by leveraging large multilingual unlabeled data.Prior teacher-student frameworks of self-training rely on rigid teaching strategies, which may hardly produce high-quality pseudo-labels for consecutive and interdependent tokens.On the contrary, MetaTS allows the teacher to dynamically adapt its pseudoannotation strategies by the student's feedback on the generated pseudo-labeled data of each language and thus mitigate error propagation from noisy pseudo-labels.Extensive experiments on both public and real-world multilingual sequence labeling datasets empirically demonstrate the effectiveness of MetaTS 1 .